ML Research Wiki / Benchmarks / Code Generation / CodeContests

CodeContests

Code Generation Benchmark

Performance Over Time

📊 Showing 8 results | 📏 Metric: Test Set pass@1

Top Performing Models

Rank Model Paper Test Set pass@1 Date Code
1 EG-CFG (DeepSeek-V3-0324) Execution Guided Line-by-Line Code Generation 58.18 2025-06-12 📦 boazlavon/eg_cfg
2 LPW (GPT-4o) Planning-Driven Programming: A Large Language Model Programming Workflow 34.70 2024-11-21 📦 you68681/lpw
3 MapCoder (GPT-4) MapCoder: Multi-Agent Code Generation for Competitive Problem Solving 28.50 2024-05-18 📦 md-ashraful-pramanik/mapcoder 📦 Luoji-zju/Agents4PLC_release
4 CodeSim (GPT4) CODESIM: Multi-Agent Code Generation and Problem Solving through Simulation-Driven Planning and Debugging 28.40 2025-02-08 📦 kagnlp/CodeGenerator
5 MoTCoder-15B MoTCoder: Elevating Large Language Models with Modular of Thought for Challenging Programming Tasks 26.34 2023-12-26 📦 dvlab-research/motcoder
6 MoTCoder-7B-v1.5 MoTCoder: Elevating Large Language Models with Modular of Thought for Challenging Programming Tasks 20.77 2023-12-26 📦 dvlab-research/motcoder
7 CodeChain + WizardCoder-15B CodeChain: Towards Modular Code Generation Through Chain of Self-revisions with Representative Sub-modules 2.35 2023-10-13 📦 SalesforceAIResearch/CodeChain
8 WizardCoder-15B WizardCoder: Empowering Code Large Language Models with Evol-Instruct 1.11 2023-06-14 📦 nlpxucan/wizardlm 📦 nickrosh/evol-teacher 📦 kyle-lyu/data-efficient-finetuning 📦 kyle-lyu/codeact

All Papers (8)

boazlavon/eg_cfg
you68681/lpw
md-ashraful-pramanik/mapcoder Luoji-zju/Agents4PLC_release
kagnlp/CodeGenerator
dvlab-research/motcoder
dvlab-research/motcoder
SalesforceAIResearch/CodeChain
nlpxucan/wizardlm nickrosh/evol-teacher