ML Research Wiki / Benchmarks / Natural Language Inference / WNLI

WNLI

Natural Language Inference Benchmark

Performance Over Time

📊 Showing 22 results | 📏 Metric: Accuracy

Top Performing Models

Rank	Model	Paper	Accuracy	Date	Code
1	DeBERTa	DeBERTa: Decoding-enhanced BERT with Disentangled Attention	94.50	2020-06-05	📦 huggingface/transformers 📦 microsoft/DeBERTa 📦 osu-nlp-group/mind2web
2	T5-XXL 11B	Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer	93.20	2019-10-23	📦 huggingface/transformers 📦 PaddlePaddle/PaddleNLP 📦 google-research/text-to-text-transfer-transformer
3	XLNet	XLNet: Generalized Autoregressive Pretraining for Language Understanding	92.50	2019-06-19	📦 huggingface/transformers 📦 PaddlePaddle/PaddleNLP 📦 zihangdai/xlnet
4	ALBERT	ALBERT: A Lite BERT for Self-supervised Learning of Language Representations	91.80	2019-09-26	📦 huggingface/transformers 📦 tensorflow/models 📦 PaddlePaddle/PaddleNLP
5	T5-XL 3B	Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer	89.70	2019-10-23	📦 huggingface/transformers 📦 PaddlePaddle/PaddleNLP 📦 google-research/text-to-text-transfer-transformer
6	StructBERTRoBERTa ensemble	StructBERT: Incorporating Language Structures into Pre-training for Deep Language Understanding	89.70	2019-08-13	-
7	HNNensemble	A Hybrid Neural Network Model for Commonsense Reasoning	89.00	2019-07-27	📦 namisan/mt-dnn 📦 microsoft/MT-DNN 📦 chunhuililili/mt_dnn
8	RoBERTa (ensemble)	RoBERTa: A Robustly Optimized BERT Pretraining Approach	89.00	2019-07-26	📦 huggingface/transformers 📦 pytorch/fairseq 📦 PaddlePaddle/PaddleNLP
9	T5-Large 770M	Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer	85.60	2019-10-23	📦 huggingface/transformers 📦 PaddlePaddle/PaddleNLP 📦 google-research/text-to-text-transfer-transformer
10	HNN	A Hybrid Neural Network Model for Commonsense Reasoning	83.60	2019-07-27	📦 namisan/mt-dnn 📦 microsoft/MT-DNN 📦 chunhuililili/mt_dnn

All Papers (22)

DeBERTa: Decoding-enhanced BERT with Disentangled Attention

2020

DeBERTa

huggingface/transformers microsoft/DeBERTa

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

2019

T5-XXL 11B

huggingface/transformers PaddlePaddle/PaddleNLP

XLNet: Generalized Autoregressive Pretraining for Language Understanding

2019

XLNet

huggingface/transformers PaddlePaddle/PaddleNLP

ALBERT: A Lite BERT for Self-supervised Learning of Language Representations

2019

ALBERT

huggingface/transformers tensorflow/models

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

2019

T5-XL 3B

huggingface/transformers PaddlePaddle/PaddleNLP

StructBERT: Incorporating Language Structures into Pre-training for Deep Language Understanding

2019

StructBERTRoBERTa ensemble

A Hybrid Neural Network Model for Commonsense Reasoning

2019

HNNensemble

namisan/mt-dnn microsoft/MT-DNN chunhuililili/mt_dnn

RoBERTa: A Robustly Optimized BERT Pretraining Approach

2019

RoBERTa (ensemble)

huggingface/transformers pytorch/fairseq

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

2019

T5-Large 770M

huggingface/transformers PaddlePaddle/PaddleNLP

A Hybrid Neural Network Model for Commonsense Reasoning

2019

HNN

namisan/mt-dnn microsoft/MT-DNN chunhuililili/mt_dnn

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

2019

T5-Base 220M

huggingface/transformers PaddlePaddle/PaddleNLP

A Surprisingly Robust Trick for Winograd Schema Challenge

2019

BERTwiki 340M (fine-tuned on WSCR)

vid-koci/bert-commonsense TangJiaLong/Knowledge-Projection-for-ERE

Finetuned Language Models Are Zero-Shot Learners

2021

FLAN 137B (zero-shot)

hiyouga/llama-efficient-tuning bigcode-project/starcoder

A Surprisingly Robust Trick for Winograd Schema Challenge

2019

BERT-large 340M (fine-tuned on WSCR)

vid-koci/bert-commonsense TangJiaLong/Knowledge-Projection-for-ERE

A Surprisingly Robust Trick for Winograd Schema Challenge

2019

BERT-base 110M (fine-tuned on WSCR)

vid-koci/bert-commonsense TangJiaLong/Knowledge-Projection-for-ERE

Finetuned Language Models Are Zero-Shot Learners

2021

FLAN 137B (few-shot, k=4)

hiyouga/llama-efficient-tuning bigcode-project/starcoder

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

2019

T5-Small 60M

huggingface/transformers PaddlePaddle/PaddleNLP

ERNIE 2.0: A Continual Pre-training Framework for Language Understanding

2019

ERNIE 2.0 Large

PaddlePaddle/PaddleNLP PaddlePaddle/ERNIE DataScienceNigeria/ERNIE-2.0-from-Baidu-Inc.

SqueezeBERT: What can computer vision teach NLP about efficient neural networks?

2020

SqueezeBERT

huggingface/transformers huggingface/transformers

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

2018

BERT-large 340M

huggingface/transformers tensorflow/models

RWKV: Reinventing RNNs for the Transformer Era

2023

RWKV-4-Raven-14B

BlinkDL/RWKV-LM blinkdl/chatrwkv

DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter

2019

DistilBERT 66M

huggingface/transformers PaddlePaddle/PaddleNLP

WNLI

Performance Over Time

Edit Benchmark Results

Edit Result

Top Performing Models

All Papers (22)

DeBERTa: Decoding-enhanced BERT with Disentangled Attention

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

XLNet: Generalized Autoregressive Pretraining for Language Understanding

ALBERT: A Lite BERT for Self-supervised Learning of Language Representations

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

StructBERT: Incorporating Language Structures into Pre-training for Deep Language Understanding

A Hybrid Neural Network Model for Commonsense Reasoning

RoBERTa: A Robustly Optimized BERT Pretraining Approach

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

A Hybrid Neural Network Model for Commonsense Reasoning

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

A Surprisingly Robust Trick for Winograd Schema Challenge

Finetuned Language Models Are Zero-Shot Learners

A Surprisingly Robust Trick for Winograd Schema Challenge

A Surprisingly Robust Trick for Winograd Schema Challenge

Finetuned Language Models Are Zero-Shot Learners

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

ERNIE 2.0: A Continual Pre-training Framework for Language Understanding

SqueezeBERT: What can computer vision teach NLP about efficient neural networks?

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

RWKV: Reinventing RNNs for the Transformer Era

DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter

Model	Paper	Accuracy	Date
DeBERTa	DeBERTa: Decoding-enhanced BERT with Disentangled…	94.50	2020-06-05
T5-XXL 11B	Exploring the Limits of Transfer Learning with a …	93.20	2019-10-23
XLNet	XLNet: Generalized Autoregressive Pretraining for…	92.50	2019-06-19
ALBERT	ALBERT: A Lite BERT for Self-supervised Learning …	91.80	2019-09-26
T5-XL 3B	Exploring the Limits of Transfer Learning with a …	89.70	2019-10-23
StructBERTRoBERTa ensemble	StructBERT: Incorporating Language Structures int…	89.70	2019-08-13
HNNensemble	A Hybrid Neural Network Model for Commonsense Rea…	89.00	2019-07-27
RoBERTa (ensemble)	RoBERTa: A Robustly Optimized BERT Pretraining Ap…	89.00	2019-07-26
T5-Large 770M	Exploring the Limits of Transfer Learning with a …	85.60	2019-10-23
HNN	A Hybrid Neural Network Model for Commonsense Rea…	83.60	2019-07-27
T5-Base 220M	Exploring the Limits of Transfer Learning with a …	78.80	2019-10-23
BERTwiki 340M (fine-tuned on WSCR)	A Surprisingly Robust Trick for Winograd Schema C…	74.70	2019-05-15
FLAN 137B (zero-shot)	Finetuned Language Models Are Zero-Shot Learners	74.60	2021-09-03
BERT-large 340M (fine-tuned on WSCR)	A Surprisingly Robust Trick for Winograd Schema C…	71.90	2019-05-15
BERT-base 110M (fine-tuned on WSCR)	A Surprisingly Robust Trick for Winograd Schema C…	70.50	2019-05-15
FLAN 137B (few-shot, k=4)	Finetuned Language Models Are Zero-Shot Learners	70.40	2021-09-03
T5-Small 60M	Exploring the Limits of Transfer Learning with a …	69.20	2019-10-23
ERNIE 2.0 Large	ERNIE 2.0: A Continual Pre-training Framework for…	67.80	2019-07-29
SqueezeBERT	SqueezeBERT: What can computer vision teach NLP a…	65.10	2020-06-19
BERT-large 340M	BERT: Pre-training of Deep Bidirectional Transfor…	65.10	2018-10-11
RWKV-4-Raven-14B	RWKV: Reinventing RNNs for the Transformer Era	49.30	2023-05-22
DistilBERT 66M	DistilBERT, a distilled version of BERT: smaller,…	44.40	2019-10-02