ACES

A Translation Accuracy Challenge Set

Dataset Information
Modalities
Texts
Introduced
2022
License
Unknown
Homepage
Official Website
Contents

Overview

ACES a dataset consisting of 68 phenomena ranging from simple perturbations at the word/character level to more complex errors based on discourse and real-world knowledge. It can be used to evaluate a wide range of Machine Translation metrics.

Variants: ACES

Associated Benchmarks

This dataset is used in 1 benchmark:

Recent Benchmark Submissions

Task Model Paper Date
Machine Translation chrF ACES: Translation Accuracy Challenge Sets … 2022-10-27
Machine Translation metricx_xl_MQM_2020 ACES: Translation Accuracy Challenge Sets … 2022-10-27
Machine Translation HWTSC-Teacher-Sim ACES: Translation Accuracy Challenge Sets … 2022-10-27
Machine Translation MS-COMET-22 ACES: Translation Accuracy Challenge Sets … 2022-10-27
Machine Translation MS-COMET-QE-22 ACES: Translation Accuracy Challenge Sets … 2022-10-27
Machine Translation KG-BERTScore ACES: Translation Accuracy Challenge Sets … 2022-10-27
Machine Translation metricx_xl_DA_2019 ACES: Translation Accuracy Challenge Sets … 2022-10-27
Machine Translation COMET-QE ACES: Translation Accuracy Challenge Sets … 2022-10-27
Machine Translation COMET-22 ACES: Translation Accuracy Challenge Sets … 2022-10-27
Machine Translation UniTE-src ACES: Translation Accuracy Challenge Sets … 2022-10-27
Machine Translation UniTE-ref ACES: Translation Accuracy Challenge Sets … 2022-10-27
Machine Translation metricx_xxl_DA_2019 ACES: Translation Accuracy Challenge Sets … 2022-10-27
Machine Translation UniTE ACES: Translation Accuracy Challenge Sets … 2022-10-27
Machine Translation Cross-QE ACES: Translation Accuracy Challenge Sets … 2022-10-27
Machine Translation COMET-20 ACES: Translation Accuracy Challenge Sets … 2022-10-27
Machine Translation BLEURT-20 ACES: Translation Accuracy Challenge Sets … 2022-10-27
Machine Translation YiSi-1 ACES: Translation Accuracy Challenge Sets … 2022-10-27
Machine Translation BERTScore ACES: Translation Accuracy Challenge Sets … 2022-10-27
Machine Translation BLEU ACES: Translation Accuracy Challenge Sets … 2022-10-27
Machine Translation f101spBLEU ACES: Translation Accuracy Challenge Sets … 2022-10-27

Research Papers

Recent papers with results on this dataset:

External Links: