Text8

Overview edit

Variants: Text8

Associated Benchmarks

This dataset is used in 1 benchmark:

• Language Modelling - Metrics: Bit per Character (BPC), Number of params

Recent Benchmark Submissions

Research Papers

Recent papers with results on this dataset:

• Bayesian Flow Networks (2023) -
• Focus Your Attention (with Adaptive IIR Filters) (2023) -
• Long-Short Transformer: Efficient Transformers for Language and Vision (2021) -
• Pay Attention when Required (2020) -
• BP-Transformer: Modelling Long-Range Context via Binary Partitioning (2019) -
• Augmenting Self-attention with Persistent Memory (2019) -
• Discrete Flows: Invertible Generative Models of Discrete Data (2019) -
• Adaptive Attention Span in Transformers (2019) -
• Dynamic Evaluation of Transformer Language Models (2019) -
• Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context (2019) -
• Character-Level Language Modeling with Deeper Self-Attention (2018) -
• Dynamic Evaluation of Neural Sequence Models (2017) -
• Multiplicative LSTM for sequence modelling (2016) -
• Hierarchical Multiscale Recurrent Neural Networks (2016) -
• Recurrent Highway Networks (2016) -
• Recurrent Batch Normalization (2016) -
• Architectural Complexity Measures of Recurrent Neural Networks (2016) -