COmmonsense Dataset Adversarially-authored by Humans
The COmmonsense Dataset Adversarially-authored by Humans (CODAH) is an evaluation set for commonsense question-answering in the sentence completion style of SWAG. As opposed to other automatically generated NLI datasets, CODAH is adversarially constructed by humans who can view feedback from a pre-trained model and use this information to design challenging commonsense questions. It contains 2801 questions in total, and uses 5-fold cross validation for evaluation.
Source: CODAH Dataset
Image Source: https://www.aclweb.org/anthology/W19-2008.pdf
Variants: CODAH
This dataset is used in 2 benchmarks:
| Task | Model | Paper | Date |
|---|---|---|---|
| Question Answering | G-DAUG-Combo + RoBERTa-Large | Generative Data Augmentation for Commonsense … | 2020-04-24 |
| Question Answering | BERT Large | CODAH: An Adversarially Authored Question-Answer … | 2019-04-08 |
| Common Sense Reasoning | BERT Large | CODAH: An Adversarially Authored Question-Answer … | 2019-04-08 |
Recent papers with results on this dataset: