ML Research Wiki / Benchmarks / Code Generation / CoNaLa

CoNaLa

Code Generation Benchmark

Performance Over Time

📊 Showing 7 results | 📏 Metric: BLEU

Top Performing Models

Rank Model Paper BLEU Date Code
1 TranX + BERT w/mined The impact of lexical and grammatical processing on generating code from natural language 34.20 2022-02-28 📦 codegenfactors/BertranX 📦 codegenfact/BertranX
2 BERT + TAE 📚 Code Generation from Natural Language with Less Prior and More Monolingual Data 33.41 2021-01-01 📦 borealisai/code-gen-tae
3 External Knowledge With API + Reranking 📚 Incorporating External Knowledge through Pre-training for Natural Language to Code Generation 32.26 2020-04-20 📦 neulab/external-knowledge-codegen 📦 zorazrw/multilingual-conala
4 External Knowledge With API 📚 Incorporating External Knowledge through Pre-training for Natural Language to Code Generation 30.69 2020-04-20 📦 neulab/external-knowledge-codegen 📦 zorazrw/multilingual-conala
5 BART W/ Mined Reading StackOverflow Encourages Cheating: Adding Question Text Improves Extractive Code Generation 30.55 2021-06-08 📦 gabeorlanski/stackoverflow-encourages-cheating
6 BART Base Reading StackOverflow Encourages Cheating: Adding Question Text Improves Extractive Code Generation 26.24 2021-06-08 📦 gabeorlanski/stackoverflow-encourages-cheating
7 TranX TRANX: A Transition-based Neural Abstract Syntax Parser for Semantic Parsing and Code Generation 24.30 2018-10-05 📦 pcyin/tranX 📦 DeepLearnXMU/CG-RL 📦 yuxiang2/CoNaLa 📦 Pro-v-7/code-generation

All Papers (7)

codegenfactors/BertranX codegenfact/BertranX
borealisai/code-gen-tae
neulab/external-knowledge-codegen zorazrw/multilingual-conala
neulab/external-knowledge-codegen zorazrw/multilingual-conala
gabeorlanski/stackoverflow-encourages-cheating
gabeorlanski/stackoverflow-encourages-cheating
pcyin/tranX DeepLearnXMU/CG-RL