ML Research Wiki / Benchmarks / Language Modelling / C4

C4

Language Modelling Benchmark

Performance Over Time

📊 Showing 9 results | 📏 Metric: Perplexity

Top Performing Models

Rank Model Paper Perplexity Date Code
1 Primer Primer: Searching for Efficient Transformers for Language Modeling 12.35 2021-09-17 📦 labmlai/annotated_deep_learning_paper_implementations 📦 google-research/google-research 📦 lucidrains/FLASH-pytorch 📦 JunnYu/x-transformers-paddle
2 Zeropoint LLM.int8 13B (vector-wise + decomp) LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale 12.45 2022-08-15 📦 timdettmers/bitsandbytes 📦 huggingface/transformers-bloom-inference 📦 kohjingyu/fromage 📦 alextmallen/adaptive-retrieval
3 T5++ Primer: Searching for Efficient Transformers for Language Modeling 12.69 2021-09-17 📦 labmlai/annotated_deep_learning_paper_implementations 📦 google-research/google-research 📦 lucidrains/FLASH-pytorch 📦 JunnYu/x-transformers-paddle
4 Original T5 Primer: Searching for Efficient Transformers for Language Modeling 13.25 2021-09-17 📦 labmlai/annotated_deep_learning_paper_implementations 📦 google-research/google-research 📦 lucidrains/FLASH-pytorch 📦 JunnYu/x-transformers-paddle
5 LLM.float32 6.7B LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale 13.30 2022-08-15 📦 timdettmers/bitsandbytes 📦 huggingface/transformers-bloom-inference 📦 kohjingyu/fromage 📦 alextmallen/adaptive-retrieval
6 LLM.float32 2.7B LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale 14.43 2022-08-15 📦 timdettmers/bitsandbytes 📦 huggingface/transformers-bloom-inference 📦 kohjingyu/fromage 📦 alextmallen/adaptive-retrieval
7 N-Grammer 343M N-Grammer: Augmenting Transformers with latent n-grams 14.79 2022-07-13 📦 tensorflow/lingvo 📦 yiyixuxu/n-grammer-flax
8 N-Grammer 288M N-Grammer: Augmenting Transformers with latent n-grams 15.01 2022-07-13 📦 tensorflow/lingvo 📦 yiyixuxu/n-grammer-flax
9 LLM.float32 1.3B LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale 15.91 2022-08-15 📦 timdettmers/bitsandbytes 📦 huggingface/transformers-bloom-inference 📦 kohjingyu/fromage 📦 alextmallen/adaptive-retrieval

All Papers (9)

labmlai/annotated_deep_learning_paper_implementations google-research/google-research

LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale

2022
Zeropoint LLM.int8 13B (vector-wise + decomp)
timdettmers/bitsandbytes huggingface/transformers-bloom-inference
labmlai/annotated_deep_learning_paper_implementations google-research/google-research
labmlai/annotated_deep_learning_paper_implementations google-research/google-research
timdettmers/bitsandbytes huggingface/transformers-bloom-inference
timdettmers/bitsandbytes huggingface/transformers-bloom-inference
tensorflow/lingvo yiyixuxu/n-grammer-flax
tensorflow/lingvo yiyixuxu/n-grammer-flax
timdettmers/bitsandbytes huggingface/transformers-bloom-inference