ML Research Wiki / Benchmarks / Zero-Shot Learning / MedConceptsQA

MedConceptsQA

Zero-Shot Learning Benchmark

Performance Over Time

📊 Showing 12 results | 📏 Metric: Accuracy

Top Performing Models

Rank Model Paper Accuracy Date Code
1 gpt-4-0125-preview GPT-4 Technical Report 52.49 2023-03-15 📦 openai/evals 📦 shmsw25/factscore 📦 unispac/visual-adversarial-examples-jailbreak-large-language-models
2 gpt-3.5-turbo Language Models are Few-Shot Learners 37.06 2020-05-28 📦 ggml-org/llama.cpp 📦 ggerganov/llama.cpp 📦 karpathy/llm.c
3 dmis-lab/biobert-v1.1 BioBERT: a pre-trained biomedical language representation model for biomedical text mining 26.15 2019-01-25 📦 dmis-lab/biobert 📦 EmilyAlsentzer/clinicalBERT 📦 naver/biobert-pretrained
4 meta-llama/Meta-Llama-3-8B-Instruct LLaMA: Open and Efficient Foundation Language Models 25.84 2023-02-27 📦 huggingface/transformers 📦 ggml-org/llama.cpp 📦 ggerganov/llama.cpp
5 epfl-llm/meditron-7b MEDITRON-70B: Scaling Medical Pretraining for Large Language Models 25.75 2023-11-27 📦 epfllm/meditron
6 dmis-lab/meerkat-7b-v1.0 Small Language Models Learn Enhanced Reasoning Skills from Medical Textbooks 25.68 2024-03-30 -
7 HuggingFaceH4/zephyr-7b-beta Zephyr: Direct Distillation of LM Alignment 25.54 2023-10-25 📦 huggingface/alignment-handbook 📦 Savannah120/alignment-handbook-PoFT
8 epfl-llm/meditron-70b MEDITRON-70B: Scaling Medical Pretraining for Large Language Models 25.36 2023-11-27 📦 epfllm/meditron
9 yikuan8/Clinical-Longformer Clinical-Longformer and Clinical-BigBird: Transformers for long clinical sequences 25.04 2022-01-27 📦 luoyuanlab/clinical-longformer
10 UFNLP/gatortron-medium GatorTron: A Large Clinical Language Model to Unlock Patient Information from Unstructured Electronic Health Records 24.86 2022-02-02 -

All Papers (12)

GPT-4 Technical Report

2023
gpt-4-0125-preview
openai/evals shmsw25/factscore
ggml-org/llama.cpp ggerganov/llama.cpp
dmis-lab/biobert EmilyAlsentzer/clinicalBERT

LLaMA: Open and Efficient Foundation Language Models

2023
meta-llama/Meta-Llama-3-8B-Instruct
huggingface/transformers ggml-org/llama.cpp
epfllm/meditron

Zephyr: Direct Distillation of LM Alignment

2023
HuggingFaceH4/zephyr-7b-beta
huggingface/alignment-handbook Savannah120/alignment-handbook-PoFT
epfllm/meditron
luoyuanlab/clinical-longformer
pharmolix/openbiomed
biomistral/biomistral