ML Research Wiki / Benchmarks / Chart Question Answering / ChartQA

ChartQA

Chart Question Answering Benchmark

Performance Over Time

📊 Showing 27 results | 📏 Metric: 1:1 Accuracy

Top Performing Models

Rank Model Paper 1:1 Accuracy Date Code
1 ChartPaLI-5B + PaLM 2-S 📚 Chart-based Reasoning: Transferring Capabilities from LLMs to VLMs 81.30 2024-03-19 -
2 Gemini Ultra Gemini: A Family of Highly Capable Multimodal Models 80.80 2023-12-19 📦 valdecy/pybibx
3 DePlot+FlanPaLM+Codex (PoT Self-Consistency) DePlot: One-shot visual language reasoning by plot-to-table translation 79.30 2022-12-20 📦 huggingface/transformers
4 ChartPaLI-5B 📚 Chart-based Reasoning: Transferring Capabilities from LLMs to VLMs 77.30 2024-03-19 -
5 DePlot+Codex (PoT Self-Consistency) DePlot: One-shot visual language reasoning by plot-to-table translation 76.70 2022-12-20 📦 huggingface/transformers
6 ScreenAI 5B (4.62 B params, w/ OCR) 📚 ScreenAI: A Vision-Language Model for UI and Infographics Understanding 76.70 2024-02-07 📦 google-research-datasets/screen_qa 📦 google-research-datasets/screen_annotation
7 SMoLA-PaLI-X Specialist Model 📚 Omni-SMoLA: Boosting Generalist Multimodal Models with Soft Mixture of Low-rank Experts 74.60 2023-12-01 -
8 SMoLA-PaLI-X Generalist Model 📚 Omni-SMoLA: Boosting Generalist Multimodal Models with Soft Mixture of Low-rank Experts 73.80 2023-12-01 -
9 MatCha4096 + LaMenDa 📚 Synthesize Step-by-Step: Tools, Templates and LLMs as Data Generators for Reasoning-Based Chart VQA 72.64 2024-01-01 -
10 PaLI-X (Single-task FT w/ OCR) 📚 PaLI-X: On Scaling up a Multilingual Vision and Language Model 72.30 2023-05-29 📦 kyegomez/PALI 📦 doc-doc/NExT-OE

All Papers (27)