QVHighlights

Query-based Video Highlights

Dataset Information
Modalities
Videos, Texts
Languages
English
Introduced
2021
Homepage

Overview

The Query-based Video Highlights (QVHighlights) dataset is a dataset for detecting customized moments and highlights from videos given natural language (NL). It consists of over 10,000 YouTube videos, covering a wide range of topics, from everyday activities and travel in lifestyle vlog videos to social and political activities in news videos. Each video in the dataset is annotated with: (1) a human-written free-form NL query, (2) relevant moments in the video w.r.t. the query, and (3) five-point scale saliency scores for all query-relevant clips.

Variants: QVHighlights

Associated Benchmarks

This dataset is used in 3 benchmarks:

Recent Benchmark Submissions

Task Model Paper Date
Moment Retrieval LD-DETR LD-DETR: Loop Decoder DEtection TRansformer … 2025-01-18
Moment Retrieval LA-DETR Length-Aware DETR for Robust Moment … 2024-12-30
Highlight Detection FlashVTG FlashVTG: Feature Layering and Adaptive … 2024-12-18
Moment Retrieval FlashVTG FlashVTG: Feature Layering and Adaptive … 2024-12-18
Moment Retrieval VideoLights-B-pt VideoLights: Feature Refinement and Cross-Task … 2024-12-02
Highlight Detection VideoLights-B-pt VideoLights: Feature Refinement and Cross-Task … 2024-12-02
Moment Retrieval LLaVA-MR LLaVA-MR: Large Language-and-Vision Assistant for … 2024-11-21
Highlight Detection NumPro Number it: Temporal Grounding Videos … 2024-11-15
Highlight Detection VideoChat-T (FT) TimeSuite: Improving MLLMs for Long … 2024-10-25
Highlight Detection SG-DETR (w/ PT) Saliency-Guided DETR for Moment Retrieval … 2024-10-02
Moment Retrieval SG-DETR Saliency-Guided DETR for Moment Retrieval … 2024-10-02
Highlight Detection SG-DETR Saliency-Guided DETR for Moment Retrieval … 2024-10-02
Moment Retrieval SG-DETR (w/ PT) Saliency-Guided DETR for Moment Retrieval … 2024-10-02
Moment Retrieval LLMEPET Prior Knowledge Integration via LLM … 2024-07-21
Highlight Detection LLMEPET Prior Knowledge Integration via LLM … 2024-07-21
Video Grounding LLMEPET Prior Knowledge Integration via LLM … 2024-07-21
Highlight Detection HL-CLIP Unleash the Potential of CLIP … 2024-04-02
Moment Retrieval R^2-Tuning $R^2$-Tuning: Efficient Image-to-Video Transfer Learning … 2024-03-31
Highlight Detection R^2-Tuning $R^2$-Tuning: Efficient Image-to-Video Transfer Learning … 2024-03-31
Moment Retrieval InternVideo2-6B InternVideo2: Scaling Foundation Models for … 2024-03-22

Research Papers

Recent papers with results on this dataset: