ML Research Wiki / Benchmarks / Code Completion / SAFIM

SAFIM

Code Completion Benchmark

Performance Over Time

📊 Showing 15 results | 📏 Metric: Average

Top Performing Models

Rank	Model	Paper	Average	Date	Code
1	deepseek-coder-33b-base	Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks	75.16	2024-03-07	📦 gonglinyuan/safim
2	deepseek-coder-6.7b-base	Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks	69.68	2024-03-07	📦 gonglinyuan/safim
3	starcoderbase	Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks	68.06	2024-03-07	📦 gonglinyuan/safim
4	gpt-4-1106-preview	Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks	62.58	2024-03-07	📦 gonglinyuan/safim
5	deepseek-coder-1.3b-base	Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks	62.58	2024-03-07	📦 gonglinyuan/safim
6	CodeLlama-13b-hf	Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks	59.68	2024-03-07	📦 gonglinyuan/safim
7	CodeLlama-34b-hf	Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks	56.45	2024-03-07	📦 gonglinyuan/safim
8	gpt-3.5-turbo-0301	Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks	53.87	2024-03-07	📦 gonglinyuan/safim
9	incoder-6B	Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks	48.06	2024-03-07	📦 gonglinyuan/safim
10	CodeLlama-7b-hf	Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks	46.77	2024-03-07	📦 gonglinyuan/safim

All Papers (15)

Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks

2024

deepseek-coder-33b-base

gonglinyuan/safim

Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks

2024

deepseek-coder-6.7b-base

gonglinyuan/safim

Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks

2024

starcoderbase

gonglinyuan/safim

Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks

2024

gpt-4-1106-preview

gonglinyuan/safim

Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks

2024

deepseek-coder-1.3b-base

gonglinyuan/safim

Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks

2024

CodeLlama-13b-hf

gonglinyuan/safim

Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks

2024

CodeLlama-34b-hf

gonglinyuan/safim

Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks

2024

gpt-3.5-turbo-0301

gonglinyuan/safim

Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks

2024

incoder-6B

gonglinyuan/safim

Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks

2024

CodeLlama-7b-hf

gonglinyuan/safim

Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks

2024

incoder-1B

gonglinyuan/safim

Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks

2024

codegen-2B-multi

gonglinyuan/safim

Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks

2024

codegen-16B-multi

gonglinyuan/safim

Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks

2024

codegen-6B-multi

gonglinyuan/safim

Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks

2024

codegen-350M-multi

gonglinyuan/safim

SAFIM

Performance Over Time

Edit Benchmark Results

Edit Result

Top Performing Models

All Papers (15)

Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks

Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks

Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks

Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks

Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks

Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks

Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks

Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks

Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks

Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks

Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks

Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks

Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks

Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks

Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks

Model	Paper	Average	Date
deepseek-coder-33b-base	Evaluation of LLMs on Syntax-Aware Code Fill-in-t…	75.16	2024-03-07
deepseek-coder-6.7b-base	Evaluation of LLMs on Syntax-Aware Code Fill-in-t…	69.68	2024-03-07
starcoderbase	Evaluation of LLMs on Syntax-Aware Code Fill-in-t…	68.06	2024-03-07
gpt-4-1106-preview	Evaluation of LLMs on Syntax-Aware Code Fill-in-t…	62.58	2024-03-07
deepseek-coder-1.3b-base	Evaluation of LLMs on Syntax-Aware Code Fill-in-t…	62.58	2024-03-07
CodeLlama-13b-hf	Evaluation of LLMs on Syntax-Aware Code Fill-in-t…	59.68	2024-03-07
CodeLlama-34b-hf	Evaluation of LLMs on Syntax-Aware Code Fill-in-t…	56.45	2024-03-07
gpt-3.5-turbo-0301	Evaluation of LLMs on Syntax-Aware Code Fill-in-t…	53.87	2024-03-07
incoder-6B	Evaluation of LLMs on Syntax-Aware Code Fill-in-t…	48.06	2024-03-07
CodeLlama-7b-hf	Evaluation of LLMs on Syntax-Aware Code Fill-in-t…	46.77	2024-03-07
incoder-1B	Evaluation of LLMs on Syntax-Aware Code Fill-in-t…	43.87	2024-03-07
codegen-2B-multi	Evaluation of LLMs on Syntax-Aware Code Fill-in-t…	32.26	2024-03-07
codegen-16B-multi	Evaluation of LLMs on Syntax-Aware Code Fill-in-t…	31.29	2024-03-07
codegen-6B-multi	Evaluation of LLMs on Syntax-Aware Code Fill-in-t…	27.74	2024-03-07
codegen-350M-multi	Evaluation of LLMs on Syntax-Aware Code Fill-in-t…	26.45	2024-03-07