Clotho

Overview edit

Variants: Clotho

Associated Benchmarks

This dataset is used in 2 benchmarks:

• Audio captioning - Metrics: SPIDEr, CIDEr, SPICE, BLEU-4, METEOR, ROUGE-L, FENSE, SPIDEr-FL, Sentence-BERT
• Text to Audio Retrieval - Metrics: R@1, R@5, R@10, mAP@10

Recent Benchmark Submissions

Research Papers

Recent papers with results on this dataset:

• Enhancing Retrieval-Augmented Audio Captioning with Generation-Assisted Multimodal Querying and Progressive Learning (2024) -
• SLAM-AAC: Enhancing Audio Captioning with Paraphrasing Augmentation and CLAP-Refine through LLMs (2024) -
• Estimated Audio-Caption Correspondences Improve Language-Based Audio Retrieval (2024) -
• Enhancing Automated Audio Captioning via Large Language Models with Optimized Audio Encoding (2024) -
• InternVideo2: Scaling Foundation Models for Multimodal Video Understanding (2024) -
• Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and Dialogue Abilities (2024) -
• Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Models (2023) -
• Advancing Natural-Language Based Audio Retrieval with PaSST and Large Audio-Caption Data Sets (2023) -
• VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Dataset (2023) -
• ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalities (2023) -
• VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and Dataset (2023) -
• Audio Retrieval with Natural Language Queries: A Benchmark Study (2021) -
• Audio Retrieval with Natural Language Queries (2021) -
• The NTT DCASE2020 Challenge Task 6 system: Automated Audio Captioning with Keywords and Sentence Length Estimation (2020) -
• Audio Captioning using Gated Recurrent Units (2020) -