ML Research Wiki / Benchmarks / Constituency Parsing / Penn Treebank

Penn Treebank

Constituency Parsing Benchmark

Performance Over Time

📊 Showing 22 results | 📏 Metric: F1 score

Top Performing Models

Rank	Model	Paper	F1 score	Date	Code
1	Hashing + XLNet	To be Continuous, or to be Discrete, Those are Bits of Questions	96.43	2024-06-12	📦 speedcell4/parserker
2	SAPar + XLNet	Improving Constituency Parsing with Span Attention	96.40	2020-10-15	📦 cuhksz-nlp/SAPar
3	Label Attention Layer + HPSG + XLNet	Rethinking Self-Attention: Towards Interpretability in Neural Parsing	96.38	2019-11-10	📦 KhalilMrini/LAL-Parser 📦 kh8fb/LAL-Parser-Server
4	Attach-Juxtapose Parser + XLNet	Strongly Incremental Constituency Parsing with Graph Neural Networks	96.34	2020-10-27	📦 yzhangcs/parser 📦 princeton-vl/attach-juxtapose-parser 📦 anaezquerro/incpar
5	Head-Driven Phrase Structure Grammar Parsing (Joint) + XLNet	Head-Driven Phrase Structure Grammar Parsing on Penn Treebank	96.33	2019-07-05	📦 DoodleJZ/HPSG-Neural-Parser
6	CRF Parser + RoBERTa	Fast and Accurate Neural CRF Constituency Parsing	96.32	2020-08-09	📦 yzhangcs/parser 📦 yzhangcs/crfpar
7	Hashing + Bert	To be Continuous, or to be Discrete, Those are Bits of Questions	96.03	2024-06-12	📦 speedcell4/parserker
8	NFC + BERT-large	Investigating Non-local Features for Neural Constituency Parsing	95.92	2021-09-27	📦 ringos/nfc-parser
9	Head-Driven Phrase Structure Grammar Parsing (Joint) + BERT	Head-Driven Phrase Structure Grammar Parsing on Penn Treebank	95.84	2019-07-05	📦 DoodleJZ/HPSG-Neural-Parser
10	CRF Parser + BERT	Fast and Accurate Neural CRF Constituency Parsing	95.69	2020-08-09	📦 yzhangcs/parser 📦 yzhangcs/crfpar

All Papers (22)

To be Continuous, or to be Discrete, Those are Bits of Questions

2024

Hashing + XLNet

speedcell4/parserker

Improving Constituency Parsing with Span Attention

2020

SAPar + XLNet

cuhksz-nlp/SAPar

Rethinking Self-Attention: Towards Interpretability in Neural Parsing

2019

Label Attention Layer + HPSG + XLNet

KhalilMrini/LAL-Parser kh8fb/LAL-Parser-Server

Strongly Incremental Constituency Parsing with Graph Neural Networks

2020

Attach-Juxtapose Parser + XLNet

yzhangcs/parser princeton-vl/attach-juxtapose-parser anaezquerro/incpar

Head-Driven Phrase Structure Grammar Parsing on Penn Treebank

2019

Head-Driven Phrase Structure Grammar Parsing (Joint) + XLNet

DoodleJZ/HPSG-Neural-Parser

Fast and Accurate Neural CRF Constituency Parsing

2020

CRF Parser + RoBERTa

yzhangcs/parser yzhangcs/crfpar

To be Continuous, or to be Discrete, Those are Bits of Questions

2024

Hashing + Bert

speedcell4/parserker

Investigating Non-local Features for Neural Constituency Parsing

2021

NFC + BERT-large

ringos/nfc-parser

Head-Driven Phrase Structure Grammar Parsing on Penn Treebank

2019

Head-Driven Phrase Structure Grammar Parsing (Joint) + BERT

DoodleJZ/HPSG-Neural-Parser

Fast and Accurate Neural CRF Constituency Parsing

2020

CRF Parser + BERT

yzhangcs/parser yzhangcs/crfpar

Cloze-driven Pretraining of Self-attention Networks

2019

CNN Large + fine-tune

Generalizing Natural Language Analysis through Span-relation Representations

2019

SpanRel

jzbjyb/SpanRel neulab/cmu-multinlp jiachengli1995/jointie

Tetra-Tagging: Word-Synchronous Parsing with Linear-Time Inference

2019

Tetra Tagging

yzhangcs/parser nikitakit/tetra-tagging

Constituency Parsing with a Self-Attentive Encoder

2018

Self-attentive encoder + ELMo

nikitakit/self-attentive-parser thomaslu2000/incremental-parsing-representations

Improving Neural Parsing by Disentangling Model Combination and Reranking Effects

2017

Model combination

Direct Output Connection for a High-Rank Language Model

2018

LSTM Encoder-Decoder + LSTM-LM

nttcslab-nlp/doc_lm

In-Order Transition-based Constituent Parsing

2017

In-order

hantek/distance-parser LeonCrashCode/InOrderParser

Fast and Accurate Neural CRF Constituency Parsing

2020

CRF Parser

yzhangcs/parser yzhangcs/crfpar

What Do Recurrent Neural Network Grammars Learn About Syntax?

2016

Stack-only RNNG

clab/rnng

Attention Is All You Need

2017

Transformer

huggingface/transformers labmlai/annotated_deep_learning_paper_implementations

Grammar as a Foreign Language

2014

Semi-supervised LSTM

atpaino/deep-text-corrector kmadathil/sanskrit_parser

Recurrent Neural Network Grammars

2016

RNN Grammar

clab/rnng dpfried/rnng-bert

Penn Treebank

Performance Over Time

Edit Benchmark Results

Edit Result

Top Performing Models

All Papers (22)

To be Continuous, or to be Discrete, Those are Bits of Questions

Improving Constituency Parsing with Span Attention

Rethinking Self-Attention: Towards Interpretability in Neural Parsing

Strongly Incremental Constituency Parsing with Graph Neural Networks

Head-Driven Phrase Structure Grammar Parsing on Penn Treebank

Fast and Accurate Neural CRF Constituency Parsing

To be Continuous, or to be Discrete, Those are Bits of Questions

Investigating Non-local Features for Neural Constituency Parsing

Head-Driven Phrase Structure Grammar Parsing on Penn Treebank

Fast and Accurate Neural CRF Constituency Parsing

Cloze-driven Pretraining of Self-attention Networks

Generalizing Natural Language Analysis through Span-relation Representations

Tetra-Tagging: Word-Synchronous Parsing with Linear-Time Inference

Constituency Parsing with a Self-Attentive Encoder

Improving Neural Parsing by Disentangling Model Combination and Reranking Effects

Direct Output Connection for a High-Rank Language Model

In-Order Transition-based Constituent Parsing

Fast and Accurate Neural CRF Constituency Parsing

What Do Recurrent Neural Network Grammars Learn About Syntax?

Attention Is All You Need

Grammar as a Foreign Language

Recurrent Neural Network Grammars

Model	Paper	F1 score	Date
Hashing + XLNet	To be Continuous, or to be Discrete, Those are Bi…	96.43	2024-06-12
SAPar + XLNet	Improving Constituency Parsing with Span Attention	96.40	2020-10-15
Label Attention Layer + HPSG + XLNet	Rethinking Self-Attention: Towards Interpretabili…	96.38	2019-11-10
Attach-Juxtapose Parser + XLNet	Strongly Incremental Constituency Parsing with Gr…	96.34	2020-10-27
Head-Driven Phrase Structure Grammar Parsing (Joint) + XLNet	Head-Driven Phrase Structure Grammar Parsing on P…	96.33	2019-07-05
CRF Parser + RoBERTa	Fast and Accurate Neural CRF Constituency Parsing	96.32	2020-08-09
Hashing + Bert	To be Continuous, or to be Discrete, Those are Bi…	96.03	2024-06-12
NFC + BERT-large	Investigating Non-local Features for Neural Const…	95.92	2021-09-27
Head-Driven Phrase Structure Grammar Parsing (Joint) + BERT	Head-Driven Phrase Structure Grammar Parsing on P…	95.84	2019-07-05
CRF Parser + BERT	Fast and Accurate Neural CRF Constituency Parsing	95.69	2020-08-09
CNN Large + fine-tune	Cloze-driven Pretraining of Self-attention Networ…	95.60	2019-03-19
SpanRel	Generalizing Natural Language Analysis through Sp…	95.50	2019-11-10
Tetra Tagging	Tetra-Tagging: Word-Synchronous Parsing with Line…	95.44	2019-04-22
Self-attentive encoder + ELMo	Constituency Parsing with a Self-Attentive Encoder	95.13	2018-05-02
Model combination	Improving Neural Parsing by Disentangling Model C…	94.66	2017-07-10
LSTM Encoder-Decoder + LSTM-LM	Direct Output Connection for a High-Rank Language…	94.47	2018-08-30
In-order	In-Order Transition-based Constituent Parsing	94.20	2017-07-17
CRF Parser	Fast and Accurate Neural CRF Constituency Parsing	94.12	2020-08-09
Stack-only RNNG	What Do Recurrent Neural Network Grammars Learn A…	93.60	2016-11-17
Transformer	Attention Is All You Need	92.70	2017-06-12
Semi-supervised LSTM	Grammar as a Foreign Language	92.10	2014-12-23
RNN Grammar	Recurrent Neural Network Grammars		2016-02-25