ML Research Wiki / Benchmarks / Reading Comprehension / RACE

RACE

Reading Comprehension Benchmark

Performance Over Time

📊 Showing 24 results | 📏 Metric: Accuracy

Top Performing Models

Rank Model Paper Accuracy Date Code
1 ALBERT (Ensemble) Improving Machine Reading Comprehension with Single-choice Decision and Transfer Learning 91.40 2020-11-06 -
2 Megatron-BERT (ensemble) Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism 90.90 2019-09-17 📦 NVIDIA/Megatron-LM 📦 PaddlePaddle/PaddleNLP 📦 kingoflolz/mesh-transformer-jax
3 ALBERTxxlarge+DUMA(ensemble) DUMA: Reading Comprehension with Transposition Thinking 89.80 2020-01-26 📦 pfZhu/duma_code 📦 iamNCJ/DUMA-pytorch-lightning 📦 RookieZB/duma_implementation_by_tf2
4 Megatron-BERT Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism 89.50 2019-09-17 📦 NVIDIA/Megatron-LM 📦 PaddlePaddle/PaddleNLP 📦 kingoflolz/mesh-transformer-jax
5 DeBERTalarge DeBERTa: Decoding-enhanced BERT with Disentangled Attention 86.80 2020-06-05 📦 huggingface/transformers 📦 microsoft/DeBERTa 📦 osu-nlp-group/mind2web
6 B10-10-10 Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processing 85.70 2020-06-05 📦 huggingface/transformers 📦 laiguokun/Funnel-Transformer 📦 chfhf/funnel-paddle
7 XLNet XLNet: Generalized Autoregressive Pretraining for Language Understanding 84.00 2019-06-19 📦 huggingface/transformers 📦 PaddlePaddle/PaddleNLP 📦 zihangdai/xlnet
8 RoBERTa RoBERTa: A Robustly Optimized BERT Pretraining Approach 83.20 2019-07-26 📦 huggingface/transformers 📦 pytorch/fairseq 📦 PaddlePaddle/PaddleNLP
9 Orca 2-13B Orca 2: Teaching Small Language Models How to Reason 82.87 2023-11-18 -
10 Orca 2-7B Orca 2: Teaching Small Language Models How to Reason 80.79 2023-11-18 -

All Papers (24)

NVIDIA/Megatron-LM PaddlePaddle/PaddleNLP
pfZhu/duma_code iamNCJ/DUMA-pytorch-lightning RookieZB/duma_implementation_by_tf2
NVIDIA/Megatron-LM PaddlePaddle/PaddleNLP
huggingface/transformers microsoft/DeBERTa
huggingface/transformers laiguokun/Funnel-Transformer chfhf/funnel-paddle
huggingface/transformers PaddlePaddle/PaddleNLP
huggingface/transformers pytorch/fairseq
ggml-org/llama.cpp ggerganov/llama.cpp
huggingface/transformers ggml-org/llama.cpp
lucidrains/CoCa-pytorch lucidrains/PaLM-pytorch
huggingface/transformers ggml-org/llama.cpp
lucidrains/CoCa-pytorch lucidrains/PaLM-pytorch
huggingface/transformers ggml-org/llama.cpp
huggingface/transformers ggml-org/llama.cpp

Language Models are Few-Shot Learners

2020
GPT-3 175B (zero-shot)
ggml-org/llama.cpp ggerganov/llama.cpp
lucidrains/CoCa-pytorch lucidrains/PaLM-pytorch
yangletliu/finlora open-finance-lab/finlora
yangletliu/finlora open-finance-lab/finlora
yangletliu/finlora open-finance-lab/finlora
yangletliu/finlora open-finance-lab/finlora