ML Research Wiki / Benchmarks / Semantic Parsing / WikiTableQuestions

WikiTableQuestions

Semantic Parsing Benchmark

Performance Over Time

📊 Showing 22 results | 📏 Metric: Accuracy (Test)

Top Performing Models

Rank	Model	Paper	Accuracy (Test)	Date	Code
1	ARTEMIS-DA	ARTEMIS-DA: An Advanced Reasoning and Transformation Engine for Multi-Step Insight Synthesis in Data Analytics	80.80	2024-12-18	-
2	SynTQA (Oracle)	SynTQA: Synergistic Table-based Question Answering via Mixture of Text-to-SQL and E2E TQA	77.50	2024-09-25	📦 siyue-zhang/SynTableQA
3	TabLaP	Accurate and Regret-aware Numerical Problem Solver for Tabular Question Answering	76.60	2024-10-10	📦 yxw-11/tablap
4	Mix SC	Rethinking Tabular Data Understanding with Large Language Models	73.60	2023-12-27	📦 Leolty/tablellm
5	SynTQA (RF) 📚	SynTQA: Synergistic Table-based Question Answering via Mixture of Text-to-SQL and E2E TQA	71.60	2024-09-25	📦 siyue-zhang/SynTableQA
6	CABINET	CABINET: Content Relevance based Noise Reduction for Table Question Answering	69.10	2024-02-02	📦 sohanpatnaik106/cabinet_qa
7	NormTab+TabSQLify	NormTab: Improving Symbolic Reasoning in LLMs Through Tabular Data Normalization	68.63	2024-06-25	📦 mahadi-nahid/NormTab
8	Chain-of-Table	Chain-of-Table: Evolving Tables in the Reasoning Chain for Table Understanding	67.31	2024-01-09	📦 cyqiq/multicot 📦 google-research/chain-of-table
9	Tab-PoT	Efficient Prompting for LLM-based Generative Internet of Things	66.78	2024-06-14	-
10	SynTQA (GPT) 📚	SynTQA: Synergistic Table-based Question Answering via Mixture of Text-to-SQL and E2E TQA	65.20	2024-09-25	📦 siyue-zhang/SynTableQA

All Papers (22)

ARTEMIS-DA: An Advanced Reasoning and Transformation Engine for Multi-Step Insight Synthesis in Data Analytics

2024

ARTEMIS-DA

SynTQA: Synergistic Table-based Question Answering via Mixture of Text-to-SQL and E2E TQA

2024

SynTQA (Oracle)

siyue-zhang/SynTableQA

Accurate and Regret-aware Numerical Problem Solver for Tabular Question Answering

2024

TabLaP

yxw-11/tablap

Rethinking Tabular Data Understanding with Large Language Models

2023

Mix SC

Leolty/tablellm

SynTQA: Synergistic Table-based Question Answering via Mixture of Text-to-SQL and E2E TQA

2024

SynTQA (RF)

siyue-zhang/SynTableQA

CABINET: Content Relevance based Noise Reduction for Table Question Answering

2024

CABINET

sohanpatnaik106/cabinet_qa

NormTab: Improving Symbolic Reasoning in LLMs Through Tabular Data Normalization

2024

NormTab+TabSQLify

mahadi-nahid/NormTab

Chain-of-Table: Evolving Tables in the Reasoning Chain for Table Understanding

2024

Chain-of-Table

cyqiq/multicot google-research/chain-of-table

Efficient Prompting for LLM-based Generative Internet of Things

2024

Tab-PoT

SynTQA: Synergistic Table-based Question Answering via Mixture of Text-to-SQL and E2E TQA

2024

SynTQA (GPT)

siyue-zhang/SynTableQA

Binding Language Models in Symbolic Languages

2022

Binder

xlang-ai/binder hkunlp/binder

Large Language Models are Versatile Decomposers: Decompose Evidence and Questions for Table-based Reasoning

2023

Dater

alibabaresearch/damo-convai google-research/chain-of-table

TabSQLify: Enhancing Reasoning Capabilities of LLMs Through Table Decomposition

2024

TabSQLify (col+row)

mahadi-nahid/tabsqlify nikhilsab/h-star

LEVER: Learning to Verify Language-to-Code Generation with Execution

2023

LEVER

niansong1996/lever

OmniTab: Pretraining with Natural and Synthetic Data for Few-shot Table-based Question Answering

2022

OmniTab-Large

jzbjyb/omnitab

NormTab: Improving Symbolic Reasoning in LLMs Through Tabular Data Normalization

2024

NormTab (Targeted) + SQL

mahadi-nahid/NormTab

ReasTAP: Injecting Table Reasoning Skills During Pre-training via Synthetic Reasoning Examples

2022

ReasTAP-Large

yale-lily/reastap

TAPEX: Table Pre-training via Learning a Neural SQL Executor

2021

TAPEX-Large

microsoft/Table-Pretraining sohanpatnaik106/cabinet_qa

TaBERT: Pretraining for Joint Understanding of Textual and Tabular Data

2020

MAPO + TABERTLarge (K = 3)

facebookresearch/tabert

UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models

2022

T5-3b(UnifiedSKG)

hkunlp/unifiedskg

TAPAS: Weakly Supervised Table Parsing via Pre-training

2020

TAPAS-Large (pre-trained on SQA)

huggingface/transformers google-research/tapas

Learning Semantic Parsers from Denotations with Latent Structured Alignments and Abstract Programs

2019

Structured Attention

berlino/weaksp_em19

WikiTableQuestions

Performance Over Time

Edit Benchmark Results

Edit Result

Top Performing Models

All Papers (22)

ARTEMIS-DA: An Advanced Reasoning and Transformation Engine for Multi-Step Insight Synthesis in Data Analytics

SynTQA: Synergistic Table-based Question Answering via Mixture of Text-to-SQL and E2E TQA

Accurate and Regret-aware Numerical Problem Solver for Tabular Question Answering

Rethinking Tabular Data Understanding with Large Language Models

SynTQA: Synergistic Table-based Question Answering via Mixture of Text-to-SQL and E2E TQA

CABINET: Content Relevance based Noise Reduction for Table Question Answering

NormTab: Improving Symbolic Reasoning in LLMs Through Tabular Data Normalization

Chain-of-Table: Evolving Tables in the Reasoning Chain for Table Understanding

Efficient Prompting for LLM-based Generative Internet of Things

SynTQA: Synergistic Table-based Question Answering via Mixture of Text-to-SQL and E2E TQA

Binding Language Models in Symbolic Languages

Large Language Models are Versatile Decomposers: Decompose Evidence and Questions for Table-based Reasoning

TabSQLify: Enhancing Reasoning Capabilities of LLMs Through Table Decomposition

LEVER: Learning to Verify Language-to-Code Generation with Execution

OmniTab: Pretraining with Natural and Synthetic Data for Few-shot Table-based Question Answering

NormTab: Improving Symbolic Reasoning in LLMs Through Tabular Data Normalization

ReasTAP: Injecting Table Reasoning Skills During Pre-training via Synthetic Reasoning Examples

TAPEX: Table Pre-training via Learning a Neural SQL Executor

TaBERT: Pretraining for Joint Understanding of Textual and Tabular Data

UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models

TAPAS: Weakly Supervised Table Parsing via Pre-training

Learning Semantic Parsers from Denotations with Latent Structured Alignments and Abstract Programs

Model	Paper	Accuracy (Test)	Date
ARTEMIS-DA	ARTEMIS-DA: An Advanced Reasoning and Transformat…	80.80	2024-12-18
SynTQA (Oracle)	SynTQA: Synergistic Table-based Question Answerin…	77.50	2024-09-25
TabLaP	Accurate and Regret-aware Numerical Problem Solve…	76.60	2024-10-10
Mix SC	Rethinking Tabular Data Understanding with Large …	73.60	2023-12-27
SynTQA (RF)	SynTQA: Synergistic Table-based Question Answerin…	71.60	2024-09-25
CABINET	CABINET: Content Relevance based Noise Reduction …	69.10	2024-02-02
NormTab+TabSQLify	NormTab: Improving Symbolic Reasoning in LLMs Thr…	68.63	2024-06-25
Chain-of-Table	Chain-of-Table: Evolving Tables in the Reasoning …	67.31	2024-01-09
Tab-PoT	Efficient Prompting for LLM-based Generative Inte…	66.78	2024-06-14
SynTQA (GPT)	SynTQA: Synergistic Table-based Question Answerin…	65.20	2024-09-25
Binder	Binding Language Models in Symbolic Languages	65.00	2022-10-06
Dater	Large Language Models are Versatile Decomposers: …	64.80	2023-01-31
TabSQLify (col+row)	TabSQLify: Enhancing Reasoning Capabilities of LL…	64.70	2024-04-15
LEVER	LEVER: Learning to Verify Language-to-Code Genera…	64.60	2023-02-16
OmniTab-Large	OmniTab: Pretraining with Natural and Synthetic D…	62.50	2022-07-08
NormTab (Targeted) + SQL	NormTab: Improving Symbolic Reasoning in LLMs Thr…	61.20	2024-06-25
ReasTAP-Large	ReasTAP: Injecting Table Reasoning Skills During …	59.70	2022-10-22
TAPEX-Large	TAPEX: Table Pre-training via Learning a Neural S…	57.00	2021-07-16
MAPO + TABERTLarge (K = 3)	TaBERT: Pretraining for Joint Understanding of Te…	52.20	2020-05-17
T5-3b(UnifiedSKG)	UnifiedSKG: Unifying and Multi-Tasking Structured…	50.65	2022-01-16
TAPAS-Large (pre-trained on SQA)	TAPAS: Weakly Supervised Table Parsing via Pre-tr…	48.80	2020-04-05
Structured Attention	Learning Semantic Parsers from Denotations with L…	43.70	2019-09-09