ML Research Wiki / Benchmarks / Question Answering / HotpotQA

HotpotQA

Question Answering Benchmark

Performance Over Time

📊 Showing 22 results | 📏 Metric: JOINT-F1

Top Performing Models

Rank	Model	Paper	JOINT-F1	Date	Code
1	Beam Retrieval	End-to-End Beam Retrieval for Multi-Hop Question Answering	0.78	2023-08-17	📦 ShayekhBinIslam/openrag 📦 canghongjian/beam_retriever 📦 Alab-NII/2wikimultihop
2	BigBird-etc	Big Bird: Transformers for Longer Sequences	0.74	2020-07-28	📦 huggingface/transformers 📦 tensorflow/models 📦 PaddlePaddle/PaddleNLP
3	AISO	Adaptive Information Seeking for Open-Domain Question Answering	0.72	2021-09-14	📦 zycdev/aiso
4	Chain-of-Skills	Chain-of-Skills: A Configurable Model for Open-domain Question Answering	0.72	2023-05-04	📦 mayer123/udt-qa
5	HopRetriever + Sp-search	HopRetriever: Retrieve Hops over Wikipedia to Answer Complex Questions	0.71	2020-12-31	-
6	IRRR+	Answering Open-Domain Questions of Varying Reasoning Steps from Text	0.70	2020-10-23	📦 beerqa/irrr
7	IRRR	Answering Open-Domain Questions of Varying Reasoning Steps from Text	0.69	2020-10-23	📦 beerqa/irrr
8	Recursive Dense Retriever	Answering Complex Open-Domain Questions with Multi-Hop Dense Retrieval	0.67	2020-09-27	📦 facebookresearch/multihop_dense_retrieval
9	DDRQA	Answering Any-hop Open-domain Questions with Iterative Document Reranking	0.64	2020-09-16	-
10	Robustly Fine-tuned Graph-based Recurrent Retriever	Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering	0.61	2019-11-24	📦 AkariAsai/learning_to_retrieve_reasoning_paths 📦 AkariAsai/XORQA

All Papers (22)

End-to-End Beam Retrieval for Multi-Hop Question Answering

2023

Beam Retrieval

ShayekhBinIslam/openrag canghongjian/beam_retriever Alab-NII/2wikimultihop

Big Bird: Transformers for Longer Sequences

2020

BigBird-etc

huggingface/transformers tensorflow/models

Adaptive Information Seeking for Open-Domain Question Answering

2021

AISO

zycdev/aiso

Chain-of-Skills: A Configurable Model for Open-domain Question Answering

2023

Chain-of-Skills

mayer123/udt-qa

HopRetriever: Retrieve Hops over Wikipedia to Answer Complex Questions

2020

HopRetriever + Sp-search

Answering Open-Domain Questions of Varying Reasoning Steps from Text

2020

IRRR+

beerqa/irrr

Answering Open-Domain Questions of Varying Reasoning Steps from Text

2020

IRRR

beerqa/irrr

Answering Complex Open-Domain Questions with Multi-Hop Dense Retrieval

2020

Recursive Dense Retriever

facebookresearch/multihop_dense_retrieval

Answering Any-hop Open-domain Questions with Iterative Document Reranking

2020

DDRQA

Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering

2019

Robustly Fine-tuned Graph-based Recurrent Retriever

AkariAsai/learning_to_retrieve_reasoning_paths AkariAsai/XORQA

Hierarchical Graph Network for Multi-hop Question Answering

2019

HGN + SemanticRetrievalMRS IR

yuwfan/HGN

Dynamically Fused Graph Network for Multi-hop Reasoning

2019

DFGN

woshiyyya/DFGN-pytorch

HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering

2018

SAFSR model

hotpotqa/hotpot mangopy/direct-rag-learning

A Simple Yet Strong Pipeline for HotpotQA

2020

Quark + SemanticRetrievalMRS IR

Revealing the Importance of Semantic Retrieval for Machine Reading at Scale

2019

SemanticRetrievalMRS

easonnie/semanticRetrievalMRS dujiaxin/semanticRetrievalMRS

Answering Complex Open-domain Questions Through Iterative Query Generation

2019

GoldEn Retriever

qipeng/golden-retriever

Cognitive Graph for Multi-Hop Reading Comprehension at Scale

2019

Cognitive Graph QA

THUDM/CogQA ShaoaAllen/CogQA

Multi-Hop Paragraph Retrieval for Open-Domain Question Answering

2019

MUPPET

yairf11/MUPPET

Multi-Paragraph Reasoning with Knowledge-enhanced Graph Neural Network

2019

KGNN

Answering while Summarizing: Multi-task Learning for Multi-hop QA with Evidence Extraction

2019

QFE

HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering

2018

Baseline Model

hotpotqa/hotpot mangopy/direct-rag-learning

Multi-hop Reading Comprehension through Question Decomposition and Rescoring

2019

DecompRC

shmsw25/DecompRC aaronbae/AnaQA

HotpotQA

Performance Over Time

Edit Benchmark Results

Edit Result

Top Performing Models

All Papers (22)

End-to-End Beam Retrieval for Multi-Hop Question Answering

Big Bird: Transformers for Longer Sequences

Adaptive Information Seeking for Open-Domain Question Answering

Chain-of-Skills: A Configurable Model for Open-domain Question Answering

HopRetriever: Retrieve Hops over Wikipedia to Answer Complex Questions

Answering Open-Domain Questions of Varying Reasoning Steps from Text

Answering Open-Domain Questions of Varying Reasoning Steps from Text

Answering Complex Open-Domain Questions with Multi-Hop Dense Retrieval

Answering Any-hop Open-domain Questions with Iterative Document Reranking

Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering

Hierarchical Graph Network for Multi-hop Question Answering

Dynamically Fused Graph Network for Multi-hop Reasoning

HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering

A Simple Yet Strong Pipeline for HotpotQA

Revealing the Importance of Semantic Retrieval for Machine Reading at Scale

Answering Complex Open-domain Questions Through Iterative Query Generation

Cognitive Graph for Multi-Hop Reading Comprehension at Scale

Multi-Hop Paragraph Retrieval for Open-Domain Question Answering

Multi-Paragraph Reasoning with Knowledge-enhanced Graph Neural Network

Answering while Summarizing: Multi-task Learning for Multi-hop QA with Evidence Extraction

HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering

Multi-hop Reading Comprehension through Question Decomposition and Rescoring

Model	Paper	JOINT-F1	Date
Beam Retrieval	End-to-End Beam Retrieval for Multi-Hop Question …	0.78	2023-08-17
BigBird-etc	Big Bird: Transformers for Longer Sequences	0.74	2020-07-28
AISO	Adaptive Information Seeking for Open-Domain Ques…	0.72	2021-09-14
Chain-of-Skills	Chain-of-Skills: A Configurable Model for Open-do…	0.72	2023-05-04
HopRetriever + Sp-search	HopRetriever: Retrieve Hops over Wikipedia to Ans…	0.71	2020-12-31
IRRR+	Answering Open-Domain Questions of Varying Reason…	0.70	2020-10-23
IRRR	Answering Open-Domain Questions of Varying Reason…	0.69	2020-10-23
Recursive Dense Retriever	Answering Complex Open-Domain Questions with Mult…	0.67	2020-09-27
DDRQA	Answering Any-hop Open-domain Questions with Iter…	0.64	2020-09-16
Robustly Fine-tuned Graph-based Recurrent Retriever	Learning to Retrieve Reasoning Paths over Wikiped…	0.61	2019-11-24
HGN + SemanticRetrievalMRS IR	Hierarchical Graph Network for Multi-hop Question…	0.60	2019-11-09
DFGN	Dynamically Fused Graph Network for Multi-hop Rea…	0.60	2019-05-16
SAFSR model	HotpotQA: A Dataset for Diverse, Explainable Mult…	0.60	2018-09-25
Quark + SemanticRetrievalMRS IR	A Simple Yet Strong Pipeline for HotpotQA	0.56	2020-04-14
SemanticRetrievalMRS	Revealing the Importance of Semantic Retrieval fo…	0.48	2019-09-17
GoldEn Retriever	Answering Complex Open-domain Questions Through I…	0.39	2019-10-15
Cognitive Graph QA	Cognitive Graph for Multi-Hop Reading Comprehensi…	0.35	2019-05-14
MUPPET	Multi-Hop Paragraph Retrieval for Open-Domain Que…	0.27	2019-06-15
KGNN	Multi-Paragraph Reasoning with Knowledge-enhanced…	0.25	2019-11-06
QFE	Answering while Summarizing: Multi-task Learning …	0.23	2019-05-21
Baseline Model	HotpotQA: A Dataset for Diverse, Explainable Mult…	0.16	2018-09-25
DecompRC	Multi-hop Reading Comprehension through Question …	0.00	2019-06-07