📊 Showing 8 results | 📏 Metric: Pass@3
| Rank | Model | Paper | Pass@3 | Date | Code |
|---|---|---|---|---|---|
| 1 | Claude 3 Haiku | PECC: Problem Extraction and Coding Challenges | 27.67 | 2024-04-29 | 📦 hallerpatrick/pecc |
| 2 | GPT-3.5 Turbo | PECC: Problem Extraction and Coding Challenges | 23.75 | 2024-04-29 | 📦 hallerpatrick/pecc |
| 3 | codechat-bison | PECC: Problem Extraction and Coding Challenges | 11.39 | 2024-04-29 | 📦 hallerpatrick/pecc |
| 4 | chat-bison | PECC: Problem Extraction and Coding Challenges | 8.48 | 2024-04-29 | 📦 hallerpatrick/pecc |
| 5 | Mixtral-8x7B-Instruct | PECC: Problem Extraction and Coding Challenges | 8.35 | 2024-04-29 | 📦 hallerpatrick/pecc |
| 6 | Phi-3-mini-128k-instruct | PECC: Problem Extraction and Coding Challenges | 7.18 | 2024-04-29 | 📦 hallerpatrick/pecc |
| 7 | WizardLM-2-7B | PECC: Problem Extraction and Coding Challenges | 3.72 | 2024-04-29 | 📦 hallerpatrick/pecc |
| 8 | Llama-3-8B-Instruct | PECC: Problem Extraction and Coding Challenges | 3.10 | 2024-04-29 | 📦 hallerpatrick/pecc |