ML Research Wiki / Benchmarks / Question Answering / StoryCloze

StoryCloze

Question Answering Benchmark

Performance Over Time

📊 Showing 20 results | 📏 Metric: Accuracy

Top Performing Models

Rank	Model	Paper	Accuracy	Date	Code
1	BLOOMZ	Crosslingual Generalization through Multitask Finetuning	96.30	2022-11-03	📦 bigscience-workshop/xmtf
2	Flipped-3B	Guess the Instruction! Flipped Learning Makes Language Models Stronger Zero-Shot Learners	95.88	2022-10-06	📦 seonghyeonye/flipped-learning
3	FLAN 137B (few-shot, k=10)	Finetuned Language Models Are Zero-Shot Learners	94.70	2021-09-03	📦 hiyouga/llama-efficient-tuning 📦 bigcode-project/starcoder 📦 bigscience-workshop/promptsource
4	T0-3B (CoT fine-tuned)	The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuning	94.50	2023-05-23	📦 kaistai/cot-collection 📦 kaist-lklab/cot-collection
5	KiC-770M	Knowledge-in-Context: Towards Knowledgeable Semi-Parametric Language Models	94.40	2022-10-28	-
6	FLAN 137B (zero-shot)	Finetuned Language Models Are Zero-Shot Learners	93.40	2021-09-03	📦 hiyouga/llama-efficient-tuning 📦 bigcode-project/starcoder 📦 bigscience-workshop/promptsource
7	Reading Strategies Model	Improving Machine Reading Comprehension with General Reading Strategies	88.30	2018-10-31	📦 nlpdata/strategy
8	RoE-3B	Exploring the Benefits of Training Expert Language Models over Instruction Tuning	86.33	2023-02-07	📦 joeljang/rlphf 📦 joeljang/elm
9	OPT-175B	SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot	79.82	2023-01-02	📦 nvidia/tensorrt-model-optimizer 📦 ist-daslab/sparsegpt 📦 nvlabs/maskllm
10	SparseGPT (175B, 50% Sparsity)	SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot	78.87	2023-01-02	📦 nvidia/tensorrt-model-optimizer 📦 ist-daslab/sparsegpt 📦 nvlabs/maskllm

All Papers (20)

Crosslingual Generalization through Multitask Finetuning

2022

BLOOMZ

bigscience-workshop/xmtf

Guess the Instruction! Flipped Learning Makes Language Models Stronger Zero-Shot Learners

2022

Flipped-3B

seonghyeonye/flipped-learning

Finetuned Language Models Are Zero-Shot Learners

2021

FLAN 137B (few-shot, k=10)

hiyouga/llama-efficient-tuning bigcode-project/starcoder

The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuning

2023

T0-3B (CoT fine-tuned)

kaistai/cot-collection kaist-lklab/cot-collection

Knowledge-in-Context: Towards Knowledgeable Semi-Parametric Language Models

2022

KiC-770M

Finetuned Language Models Are Zero-Shot Learners

2021

FLAN 137B (zero-shot)

hiyouga/llama-efficient-tuning bigcode-project/starcoder

Improving Machine Reading Comprehension with General Reading Strategies

2018

Reading Strategies Model

nlpdata/strategy

Exploring the Benefits of Training Expert Language Models over Instruction Tuning

2023

RoE-3B

joeljang/rlphf joeljang/elm

SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot

2023

OPT-175B

nvidia/tensorrt-model-optimizer ist-daslab/sparsegpt

SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot

2023

SparseGPT (175B, 50% Sparsity)

nvidia/tensorrt-model-optimizer ist-daslab/sparsegpt

SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot

2023

SparseGPT (175B, 4:8 Sparsity)

nvidia/tensorrt-model-optimizer ist-daslab/sparsegpt

A Simple and Effective Approach to the Story Cloze Test

2018

val-LS-skip

SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot

2023

SparseGPT (175B, 2:4 Sparsity)

nvidia/tensorrt-model-optimizer ist-daslab/sparsegpt

Efficient Language Modeling with Sparse all-MLP

2022

sMLP – deterministic 9.4B (0-shot)

Efficient Language Modeling with Sparse all-MLP

2022

Switch Transformer 9B

Language Models are Few-Shot Learners

2020

GPT-3 Large 760M (zero-shot)

ggml-org/llama.cpp ggerganov/llama.cpp

Efficient Language Modeling with Sparse all-MLP

2022

Gshard 9B

Efficient Language Modeling with Sparse all-MLP

2022

HASH Layers 10B (0-shot)

Efficient Language Modeling with Sparse all-MLP

2022

Base Layers 10B (0-shot)

SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot

2023

OPT-175B (50% Sparsity)

nvidia/tensorrt-model-optimizer ist-daslab/sparsegpt

StoryCloze

Performance Over Time

Edit Benchmark Results

Edit Result

Top Performing Models

All Papers (20)

Crosslingual Generalization through Multitask Finetuning

Guess the Instruction! Flipped Learning Makes Language Models Stronger Zero-Shot Learners

Finetuned Language Models Are Zero-Shot Learners

The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuning

Knowledge-in-Context: Towards Knowledgeable Semi-Parametric Language Models

Finetuned Language Models Are Zero-Shot Learners

Improving Machine Reading Comprehension with General Reading Strategies

Exploring the Benefits of Training Expert Language Models over Instruction Tuning

SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot

SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot

SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot

A Simple and Effective Approach to the Story Cloze Test

SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot

Efficient Language Modeling with Sparse all-MLP

Efficient Language Modeling with Sparse all-MLP

Language Models are Few-Shot Learners

Efficient Language Modeling with Sparse all-MLP

Efficient Language Modeling with Sparse all-MLP

Efficient Language Modeling with Sparse all-MLP

SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot

Model	Paper	Accuracy	Date
BLOOMZ	Crosslingual Generalization through Multitask Fin…	96.30	2022-11-03
Flipped-3B	Guess the Instruction! Flipped Learning Makes Lan…	95.88	2022-10-06
FLAN 137B (few-shot, k=10)	Finetuned Language Models Are Zero-Shot Learners	94.70	2021-09-03
T0-3B (CoT fine-tuned)	The CoT Collection: Improving Zero-shot and Few-s…	94.50	2023-05-23
KiC-770M	Knowledge-in-Context: Towards Knowledgeable Semi-…	94.40	2022-10-28
FLAN 137B (zero-shot)	Finetuned Language Models Are Zero-Shot Learners	93.40	2021-09-03
Reading Strategies Model	Improving Machine Reading Comprehension with Gene…	88.30	2018-10-31
RoE-3B	Exploring the Benefits of Training Expert Languag…	86.33	2023-02-07
OPT-175B	SparseGPT: Massive Language Models Can Be Accurat…	79.82	2023-01-02
SparseGPT (175B, 50% Sparsity)	SparseGPT: Massive Language Models Can Be Accurat…	78.87	2023-01-02
SparseGPT (175B, 4:8 Sparsity)	SparseGPT: Massive Language Models Can Be Accurat…	77.02	2023-01-02
val-LS-skip	A Simple and Effective Approach to the Story Cloz…	76.50	2018-03-15
SparseGPT (175B, 2:4 Sparsity)	SparseGPT: Massive Language Models Can Be Accurat…	76.19	2023-01-02
sMLP – deterministic 9.4B (0-shot)	Efficient Language Modeling with Sparse all-MLP	74.70	2022-03-14
Switch Transformer 9B	Efficient Language Modeling with Sparse all-MLP	73.30	2022-03-14
GPT-3 Large 760M (zero-shot)	Language Models are Few-Shot Learners	72.40	2020-05-28
Gshard 9B	Efficient Language Modeling with Sparse all-MLP	67.90	2022-03-14
HASH Layers 10B (0-shot)	Efficient Language Modeling with Sparse all-MLP	64.70	2022-03-14
Base Layers 10B (0-shot)	Efficient Language Modeling with Sparse all-MLP	61.40	2022-03-14
OPT-175B (50% Sparsity)	SparseGPT: Massive Language Models Can Be Accurat…	47.10	2023-01-02