ML Research Wiki / Benchmarks / Audio-Visual Speech Recognition / LRS2

LRS2

Audio-Visual Speech Recognition Benchmark

Performance Over Time

📊 Showing 8 results | 📏 Metric: Test WER

Top Performing Models

Rank Model Paper Test WER Date Code
1 TM-Seq2seq Deep Audio-Visual Speech Recognition 8.50 2018-09-06 📦 lordmartian/deep_avsr 📦 smeetrs/deep_avsr 📦 exgc/avmust-ted 📦 amitai1992/AutomatedLipReading
2 TM-CTC Deep Audio-Visual Speech Recognition 8.20 2018-09-06 📦 lordmartian/deep_avsr 📦 smeetrs/deep_avsr 📦 exgc/avmust-ted 📦 amitai1992/AutomatedLipReading
3 CTC/Attention Audio-Visual Speech Recognition With A Hybrid CTC/Attention Architecture 7.00 2018-09-28 -
4 LF-MMI TDNN Audio-visual Recognition of Overlapped speech for the LRS2 dataset 5.90 2020-01-06 -
5 End2end Conformer End-to-end Audio-visual Speech Recognition with Conformers 3.70 2021-02-12 📦 zziz/pwc 📦 mpc001/Visual_Speech_Recognition_for_Multiple_Languages 📦 mpc001/auto_avsr
6 MoCo + wav2vec (w/o extLM) Leveraging Unimodal Self-Supervised Learning for Multimodal Audio-Visual Speech Recognition 2.60 2022-02-24 📦 lumia-group/leveraging-self-supervised-learning-for-avsr
7 CTC/Attention 📚 Auto-AVSR: Audio-Visual Speech Recognition with Automatic Labels 1.50 2023-03-25 📦 mpc001/auto_avsr 📦 umbertocappellazzo/llama-avsr
8 Whisper-Flamingo 📚 Whisper-Flamingo: Integrating Visual Features into Whisper for Audio-Visual Speech Recognition and Translation 1.40 2024-06-14 📦 roudimit/whisper-flamingo

All Papers (8)

lordmartian/deep_avsr smeetrs/deep_avsr
lordmartian/deep_avsr smeetrs/deep_avsr
zziz/pwc mpc001/Visual_Speech_Recognition_for_Multiple_Languages mpc001/auto_avsr
lumia-group/leveraging-self-supervised-learning-for-avsr
mpc001/auto_avsr umbertocappellazzo/llama-avsr
roudimit/whisper-flamingo