Situations With Adversarial Generations
Given a partial description like "she opened the hood of the car," humans can reason about the situation and anticipate what might come next ("then, she examined the engine"). SWAG (Situations With Adversarial Generations) is a large-scale dataset for this task of grounded commonsense inference, unifying natural language inference and physically grounded reasoning.
The dataset consists of 113k multiple choice questions about grounded situations. Each question is a video caption from LSMDC or ActivityNet Captions, with four answer choices about what might happen next in the scene. The correct answer is the (real) video caption for the next event in the video; the three incorrect answers are adversarially generated and human verified, so as to fool machines but not humans. The authors aim for SWAG to be a benchmark for evaluating grounded commonsense NLI and for learning representations.
Source: SWAG
Image Source: Zellers et al
Variants: SWAG
This dataset is used in 2 benchmarks:
Task | Model | Paper | Date |
---|---|---|---|
Question Answering | DeBERTaV3large | DeBERTaV3: Improving DeBERTa using ELECTRA-Style … | 2021-11-18 |
Common Sense Reasoning | DeBERTalarge | DeBERTa: Decoding-enhanced BERT with Disentangled … | 2020-06-05 |
Common Sense Reasoning | RoBERTa | RoBERTa: A Robustly Optimized BERT … | 2019-07-26 |
Common Sense Reasoning | BERT-LARGE | BERT: Pre-training of Deep Bidirectional … | 2018-10-11 |
Common Sense Reasoning | ESIM + ELMo | SWAG: A Large-Scale Adversarial Dataset … | 2018-08-16 |
Common Sense Reasoning | ESIM + GloVe | SWAG: A Large-Scale Adversarial Dataset … | 2018-08-16 |
Recent papers with results on this dataset: