COPA

Choice of Plausible Alternatives

Dataset Information
Modalities
Texts
Languages
English
Introduced
2011
Homepage

Overview

The Choice Of Plausible Alternatives (COPA) evaluation provides researchers with a tool for assessing progress in open-domain commonsense causal reasoning. COPA consists of 1000 questions, split equally into development and test sets of 500 questions each. Each question is composed of a premise and two alternatives, where the task is to select the alternative that more plausibly has a causal relation with the premise. The correct alternative is randomized so that the expected performance of randomly guessing is 50%.

Source: Choice of Plausible Alternatives (COPA)
Image Source: https://people.ict.usc.edu/~gordon/copa.html

Variants: COPA

Associated Benchmarks

This dataset is used in 1 benchmark:

Recent Benchmark Submissions

Task Model Paper Date
Question Answering T0-3B (CoT fine-tuned) The CoT Collection: Improving Zero-shot … 2023-05-23
Question Answering PaLM 2-L (1-shot) PaLM 2 Technical Report 2023-05-17
Question Answering PaLM 2-M (1-shot) PaLM 2 Technical Report 2023-05-17
Question Answering PaLM 2-S (1-shot) PaLM 2 Technical Report 2023-05-17
Question Answering OPT 66B (one-shot) BloombergGPT: A Large Language Model … 2023-03-30
Question Answering Bloomberg GPT (one-shot) BloombergGPT: A Large Language Model … 2023-03-30
Question Answering BLOOM 176B (one-shot) BloombergGPT: A Large Language Model … 2023-03-30
Question Answering GPT-NeoX (one-shot) BloombergGPT: A Large Language Model … 2023-03-30
Question Answering RoE-3B Exploring the Benefits of Training … 2023-02-07
Question Answering Hybrid H3 2.7B (0-shot, logit scoring) Hungry Hungry Hippos: Towards Language … 2022-12-28
Question Answering Hybrid H3 125M (0-shot, logit scoring) Hungry Hungry Hippos: Towards Language … 2022-12-28
Question Answering Hybrid H3 125M (0-shot, rank classification) Hungry Hungry Hippos: Towards Language … 2022-12-28
Question Answering H3 125M (0-shot, rank classification) Hungry Hungry Hippos: Towards Language … 2022-12-28
Question Answering Hybrid H3 2.7B (3-shot, logit scoring) Hungry Hungry Hippos: Towards Language … 2022-12-28
Question Answering Turing NLR v5 XXL 5.4B (fine-tuned) Toward Efficient Language Model Pretraining … 2022-12-04
Question Answering Vega v2 6B (KD-based prompt transfer) Toward Efficient Language Model Pretraining … 2022-12-04
Question Answering KiC-770M Knowledge-in-Context: Towards Knowledgeable Semi-Parametric Language … 2022-10-28
Question Answering Flipped-3B Guess the Instruction! Flipped Learning … 2022-10-06
Question Answering Neo-6B (few-shot) Ask Me Anything: A simple … 2022-10-05
Question Answering Neo-6B (QA + WS) Ask Me Anything: A simple … 2022-10-05

Research Papers

Recent papers with results on this dataset: