ActivityNet

Overview edit

Variants: ActivityNet, ActivityNet-1.3, ActivityNet-1.2, ActivityNet-GZSL (cls), ActivityNet-GZSL(main)

Associated Benchmarks

This dataset is used in 6 benchmarks:

• Visual Question Answering (VQA) - Metrics: ClipMatch@1, ClipMatch@5, Contains, ExactMatch, Follow-up ClipMatch@1, Follow-up ClipMatch@5, Follow-up Contains, Follow-up ExactMatch
• Action Recognition - Metrics: mAP
• Video Retrieval - Metrics: text-to-video R@1, text-to-video R@5, text-to-video R@10, text-to-video R@50, text-to-video Mean Rank, text-to-video Median Rank, video-to-text R@1, video-to-text R@5, video-to-text Mean Rank, video-to-text Median Rank, video-to-text R@10, video-to-text R@50
• Zero-Shot Action Recognition - Metrics: Top-1 Accuracy
• Action Recognition In Videos - Metrics: mAP
• Zero-Shot Video Retrieval - Metrics: text-to-video R@1, text-to-video R@5, text-to-video R@10, video-to-text R@1, video-to-text R@5, video-to-text R@10

Recent Benchmark Submissions

Research Papers

Recent papers with results on this dataset:

• Gramian Multimodal Representation Learning and Alignment (2024) -
• InternVideo2: Scaling Foundation Models for Multimodal Video Understanding (2024) -
• vid-TLDR: Training Free Token merging for Light-weight Video Transformer (2024) -
• Open-ended VQA benchmarking of Vision-Language models by exploiting Classification datasets and their semantic hierarchy (2024) -
• RTQ: Rethinking Video-language Understanding Based on Image-text Model (2023) -
• TESTA: Temporal-Spatial Token Aggregation for Long-form Video-Language Understanding (2023) -
• LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignment (2023) -
• BT-Adapter: Video Conversation is Feasible Without Video Instruction Tuning (2023) -
• Dual-Modal Attention-Enhanced Text-Video Retrieval with Triplet Partial Margin Contrastive Learning (2023) -
• COSA: Concatenated Sample Pretrained Vision-Language Foundation Model (2023) -
• VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Dataset (2023) -
• VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and Dataset (2023) -
• Unmasked Teacher: Towards Training-Efficient Video Foundation Models (2023) -
• Video-Text as Game Players: Hierarchical Banzhaf Interaction for Cross-Modal Representation Learning (2023) -
• DiffusionRet: Generative Text-Video Retrieval with Diffusion Model (2023) -
• Bidirectional Cross-Modal Knowledge Exploration for Video Recognition with Pre-trained Vision-Language Models (2022) -
• HiTeA: Hierarchical Temporal-Aware Video-Language Pre-training (2022) -
• VindLU: A Recipe for Effective Video-and-Language Pretraining (2022) -
• VideoCoCa: Video-Text Modeling with Zero-Shot Transfer from Contrastive Captioners (2022) -
• InternVideo: General Video Foundation Models via Generative and Discriminative Learning (2022) -