MM-Vet

Overview edit

Variants: MM-Vet

Associated Benchmarks

This dataset is used in 2 benchmarks:

• Visual Question Answering (VQA) - Metrics: Acc
• Visual Question Answering - Metrics: GPT-4 score, Params

Recent Benchmark Submissions

Research Papers

Recent papers with results on this dataset:

• Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling (2025) -
• Parameter-Inverted Image Pyramid Networks for Visual Perception and Multimodal Understanding (2025) -
• Lyra: An Efficient and Speech-Centric Framework for Omni-Cognition (2024) -
• ProVision: Programmatically Scaling Vision-centric Instruction Data for Multimodal Language Models (2024) -
• ILLUME: Illuminating Your LLMs to See, Draw, and Self-Enhance (2024) -
• TACO: Learning Multi-modal Action Models with Synthetic Chains-of-Thought-and-Action (2024) -
• Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling (2024) -
• MAmmoTH-VL: Eliciting Multimodal Reasoning with Instruction Tuning at Scale (2024) -
• LinVT: Empower Your Image-level Large Language Model to Understand Videos (2024) -
• VisionZip: Longer is Better but Not Necessary in Vision Language Models (2024) -
• FlashSloth: Lightning Multimodal Large Language Models via Embedded Visual Compression (2024) -
• A Stitch in Time Saves Nine: Small VLM is a Precise Guidance for Accelerating Large VLMs (2024) -
• Dynamic-LLaVA: Efficient Multimodal Large Language Models via Dynamic Vision-language Context Sparsification (2024) -
• Looking Beyond Text: Reducing Language bias in Large Vision-Language Models via Multimodal Dual-Attention and Soft-Image Guidance (2024) -
• FocusLLaVA: A Coarse-to-Fine Approach for Efficient and Effective Visual Token Compression (2024) -
• JanusFlow: Harmonizing Autoregression and Rectified Flow for Unified Multimodal Understanding and Generation (2024) -
• Aligned Vector Quantization for Edge-Cloud Collabrative Vision-Language Models (2024) -
• $γ-$MoD: Exploring Mixture-of-Depth Adaptation for Multimodal Large Language Models (2024) -
• Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generation (2024) -
• Improving Multi-modal Large Language Model through Boosting Vision Capabilities (2024) -