ML Research Wiki / Benchmarks / Language Modelling / OpenWebText

OpenWebText

Language Modelling Benchmark

Performance Over Time

📊 Showing 12 results | 📏 Metric: eval_perplexity

Top Performing Models

Rank Model Paper eval_perplexity Date Code
1 SEDD Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution 24.10 2023-10-25 📦 louaaron/score-entropy-discrete-diffusion 📦 ozekri/SEPO 📦 lingxiaoshawn/usd3 📦 gabeguo/any-order-speculative-decoding
2 MDLM Simple and Effective Masked Diffusion Language Models 22.98 2024-06-11 📦 kuleshov-group/mdlm 📦 masa-ue/svdd
3 GenMD4 Simplified and Generalized Masked Diffusion for Discrete Data 21.80 2024-06-06 📦 google-deepmind/md4
4 EDLM-NCE Energy-Based Diffusion Language Models for Text Generation 21.52 2024-10-28 -
5 BD3-LMs Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models 20.73 2025-03-12 📦 kuleshov-group/bd3lms 📦 MindSpore-scientific/code-12
6 EDLM-coAR Energy-Based Diffusion Language Models for Text Generation 17.58 2024-10-28 -
7 ARM Simple and Effective Masked Diffusion Language Models 17.54 2024-06-11 📦 kuleshov-group/mdlm 📦 masa-ue/svdd
8 MDLM-Prime Beyond Masked and Unmasked: Discrete Diffusion Models via Partial Masking 15.36 2025-05-24 -
9 GPT2-GELU Polynomial, trigonometric, and tropical activations 2.95 2025-02-03 📦 K-H-Ismail/torchortho
10 GPT2-Fourier Polynomial, trigonometric, and tropical activations 2.93 2025-02-03 📦 K-H-Ismail/torchortho

All Papers (12)

louaaron/score-entropy-discrete-diffusion ozekri/SEPO
kuleshov-group/mdlm masa-ue/svdd
google-deepmind/md4
kuleshov-group/bd3lms MindSpore-scientific/code-12
kuleshov-group/mdlm masa-ue/svdd
K-H-Ismail/torchortho
K-H-Ismail/torchortho
K-H-Ismail/torchortho
K-H-Ismail/torchortho