ML Research Wiki / Benchmarks / Text-To-SQL / Spider 2.0

Spider 2.0

Text-To-SQL Benchmark

Performance Over Time

📊 Showing 8 results | 📏 Metric: Success Rate

Top Performing Models

Rank Model Paper Success Rate Date Code
1 Spider-Agent + o1-preview Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows 17.03 2024-11-12 -
2 Spider-Agent + GPT-4o Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows 10.13 2024-11-12 -
3 Spider-Agent + Claude-3.5-Sonnect Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows 9.02 2024-11-12 -
4 Spider-Agent + GPT-4 Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows 8.86 2024-11-12 -
5 Spider-Agent + Qwen2.5-72B Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows 6.17 2024-11-12 -
6 Spider-Agent + DeepSeek-V2.5 Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows 5.22 2024-11-12 -
7 Spider-Agent + Gemini-Pro-1.5 Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows 2.53 2024-11-12 -
8 Spider-Agent + Llama-3.1-405B Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows 2.21 2024-11-12 -

All Papers (8)