Depth Anything V2

Abstract edit

Summary

This paper introduces Depth Anything V2, a foundational model for monocular depth estimation (MDE) that addresses challenges in producing robust and fine-grained depth predictions in complex scenes. To achieve this, the authors propose a novel training strategy by utilizing synthetic images with precise labels and enhancing the model's capabilities through large-scale pseudo-labeled real images. They critically analyze the limitations of conventional real datasets, such as label noise and lack of detail, arguing for the superiority of synthetic data in certain contexts. The paper presents a new evaluation benchmark, DA-2K, which is designed to overcome the limitations of existing benchmarks by providing high-resolution images with precise sparse depth annotations. The results demonstrate significant improvements over previous models, confirming the effectiveness of the proposed methods and the importance of integrating large-scale unlabeled real data in MDE tasks.

Methods add

This paper employs the following methods:

• Monocular Depth Estimation (MDE)
• Discriminative Models
• Generative Models
• Knowledge Distillation

Models Used add

• Depth Anything V1
• DINOv2
• Marigold

Datasets add

The following datasets were used in this research:

• Hypersim
• Virtual KITTI
• ImageNet-21K
• Objects365
• Open Images V7
• Places365
• BDD100K
• Google Landmarks
• SA-1B
• DIML

Evaluation Metrics add

• AbsRel
• δ1
• F1-score

Results add

• Depth Anything V2 outperforms previous models in depth accuracy and inference speed.
• Achieved a competitive score of 83.6% in the Transparent Surface Challenge
• Significantly better performance on the proposed DA-2K evaluation benchmark compared to other models.

Limitations add

The authors identified the following limitations:

• Heavy computational burden due to the use of 62M unlabeled images.

Technical Requirements edit

• Number of GPUs: None specified
• GPU Type: None specified

Related Papers

Papers with similar methods or datasets:

• Semi-supervised Learning with Deep Generative Models (2014) - 2697 citations
• Generative Agents: Interactive Simulacra of Human Behavior (2023) - 1535 citations
• Zero-Shot Learning -A Comprehensive Evaluation of the Good, the Bad and the Ugly (2017) - 1507 citations
• Personalizing Dialogue Agents: I have a dog, do you have pets too? (2018) - 1395 citations
• Deep Reinforcement Learning for de-novo Drug Design (2017) - 979 citations
• Virtual Worlds as Proxy for Multi-Object Tracking Analysis (2016) - 1051 citations
• Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomization (2018) - 786 citations
• Evaluating Scalable Bayesian Deep Learning Methods for Robust Computer Vision (2019) - 290 citations
• VIRTUAL KITTI 2 A PREPRINT (2020) - 257 citations
• MultiMAE: Multi-modal Multi-task Masked Autoencoders Masked inputs MultiMAE predictions Target Semantic Depth RGB Masked inputs MultiMAE predictions Target Masked inputs MultiMAE predictions Target (2022) - 248 citations

Papers Using Similar Methods

• Semi-supervised Learning with Deep Generative Models (2014)
• Generative Agents: Interactive Simulacra of Human Behavior (2023)
• Zero-Shot Learning -A Comprehensive Evaluation of the Good, the Bad and the Ugly (2017)
• Personalizing Dialogue Agents: I have a dog, do you have pets too? (2018)
• Deep Reinforcement Learning for de-novo Drug Design (2017)

External Resources