The CommonsenseQA is a dataset for commonsense question answering task. The dataset consists of 12,247 questions with 5 choices each.
The dataset was generated by Amazon Mechanical Turk workers in the following process (an example is provided in parentheses):
- a crowd worker observes a source concept from ConceptNet (“River”) and three target concepts (“Waterfall”, “Bridge”, “Valley”) that are all related by the same ConceptNet relation (“AtLocation”),
- the worker authors three questions, one per target concept, such that only that particular target concept is the answer, while the other two distractor concepts are not, (“Where on a river can you hold a cup upright to catch water on a sunny day?”, “Where can I stand on a river to see water falling without getting wet?”, “I’m crossing the river, my feet are wet but my body is dry, where am I?”)
- for each question, another worker chooses one additional distractor from Concept Net (“pebble”, “stream”, “bank”), and the author another distractor (“mountain”, “bottom”, “island”) manually.
Source: CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge
Image Source: CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge
Variants: CommonsenseQA
This dataset is used in 1 benchmark:
Recent papers with results on this dataset: