EconLogicQA

Dataset Information
Introduced
2024
License
CC BY-NC-SA 4.0
Homepage

Overview

EconLogicQA is a benchmark designed to test the sequential reasoning skills of large language models (LLMs) in economics, business, and supply chain management. It diverges from typical benchmarks by requiring models to understand and sequence multiple interconnected events, capturing complex economic logics. The benchmark includes multi-event scenarios and a thorough suite of evaluations to assess proficiency in economic contexts.

Variants: EconLogicQA

Associated Benchmarks

This dataset is used in 1 benchmark:

Recent Benchmark Submissions

Task Model Paper Date
Sentence Ordering GPT-4-Turbo EconLogicQA: A Question-Answering Benchmark for … 2024-05-13
Sentence Ordering GPT-4 EconLogicQA: A Question-Answering Benchmark for … 2024-05-13
Sentence Ordering GPT-3.5-Turbo EconLogicQA: A Question-Answering Benchmark for … 2024-05-13
Sentence Ordering Llama-3-8B-Instruct EconLogicQA: A Question-Answering Benchmark for … 2024-05-13
Sentence Ordering Mistral-7B-Instruct-v0.2 EconLogicQA: A Question-Answering Benchmark for … 2024-05-13
Sentence Ordering Mistral-7B-v0.1 EconLogicQA: A Question-Answering Benchmark for … 2024-05-13
Sentence Ordering Mistral-7B-v0.2 EconLogicQA: A Question-Answering Benchmark for … 2024-05-13
Sentence Ordering Llama-3-8B EconLogicQA: A Question-Answering Benchmark for … 2024-05-13
Sentence Ordering Zephyr-7B-Alpha EconLogicQA: A Question-Answering Benchmark for … 2024-05-13
Sentence Ordering Yi-6B-Chat EconLogicQA: A Question-Answering Benchmark for … 2024-05-13
Sentence Ordering Zephyr-7B-Beta EconLogicQA: A Question-Answering Benchmark for … 2024-05-13
Sentence Ordering Mistral-7B-Instruct-v0.1 EconLogicQA: A Question-Answering Benchmark for … 2024-05-13
Sentence Ordering Llama-2-13B-Chat EconLogicQA: A Question-Answering Benchmark for … 2024-05-13
Sentence Ordering Llama-2-7B-Chat EconLogicQA: A Question-Answering Benchmark for … 2024-05-13
Sentence Ordering Gemma-2B-IT EconLogicQA: A Question-Answering Benchmark for … 2024-05-13
Sentence Ordering Yi-6B EconLogicQA: A Question-Answering Benchmark for … 2024-05-13
Sentence Ordering Gemma-7B-IT EconLogicQA: A Question-Answering Benchmark for … 2024-05-13
Sentence Ordering Llama-2-7B EconLogicQA: A Question-Answering Benchmark for … 2024-05-13

Research Papers

Recent papers with results on this dataset: