ML Research Wiki / Benchmarks / Audio captioning / Clotho

Clotho

Audio captioning Benchmark

Performance Over Time

📊 Showing 9 results | 📏 Metric: SPIDEr

Top Performing Models

Rank Model Paper SPIDEr Date Code
1 VAST 📚 VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Dataset 19.00 2023-05-29 📦 TXH-mercury/VALOR 📦 txh-mercury/vast
2 MQ-Cap Enhancing Retrieval-Augmented Audio Captioning with Generation-Assisted Multimodal Querying and Progressive Learning 18.10 2024-10-14 -
3 Audio Flamingo (Pengi trainset) 📚 Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and Dialogue Abilities 17.40 2024-02-02 📦 NVIDIA/audio-flamingo
4 VALOR 📚 VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and Dataset 16.20 2023-04-17 📦 TXH-mercury/VALOR
5 SLAM-AAC 📚 SLAM-AAC: Enhancing Audio Captioning with Paraphrasing Augmentation and CLAP-Refine through LLMs 0.52 2024-10-12 📦 X-LANCE/SLAM-LLM
6 LOAE 📚 Enhancing Automated Audio Captioning via Large Language Models with Optimized Audio Encoding 0.51 2024-06-19 📦 frankenliu/LOAE
7 Qwen-Audio 📚 Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Models 0.44 2023-11-14 📦 alibaba-damo-academy/FunASR 📦 qwenlm/qwen-audio
8 Ensemble The NTT DCASE2020 Challenge Task 6 system: Automated Audio Captioning with Keywords and Sentence Length Estimation 0.32 2020-07-01 -
9 RNN-GRU-EncDec + VGGish + Word2Vec Audio Captioning using Gated Recurrent Units 0.18 2020-06-05 -

All Papers (9)

TXH-mercury/VALOR txh-mercury/vast
NVIDIA/audio-flamingo
TXH-mercury/VALOR
X-LANCE/SLAM-LLM
frankenliu/LOAE
alibaba-damo-academy/FunASR qwenlm/qwen-audio

Audio Captioning using Gated Recurrent Units

2020
RNN-GRU-EncDec + VGGish + Word2Vec