← ML Research Wiki / 2308.04079

3D Gaussian Splatting for Real-Time Radiance Field Rendering

Bernhard Kerbl [email protected] Inria Université Côte d'Azur France GEORGIOS KOPANAS * Inria Université Côte d'Azur France Max-Planck-Institut für Informatik THOMAS LEIMKÜHLER Germany GEOR (2023)

Paper Information
arXiv ID
Venue
ACM Transactions on Graphics
Domain
computer vision, computer graphics
SOTA Claim
Yes
Reproducibility
7/10

Abstract

Abstract not available.

Summary

The paper presents a novel method for real-time radiance field rendering using a 3D Gaussian representation. The authors introduce three key innovations: a 3D Gaussian scene representation, an optimization technique for Gaussian properties, and a fast visibility-aware rendering algorithm. Their approach achieves competitive training speeds and state-of-the-art visual quality at real-time rendering rates of 93 frames per second at 1080p resolution, with a significant improvement in efficiency compared to previous state-of-the-art methods like Mip-NeRF360 and InstantNGP. The method is tested on a variety of established datasets demonstrating its effectiveness in creating high-quality novel views from captured scene images while ensuring lower training times.

Methods

This paper employs the following methods:

  • 3D Gaussian representation
  • differentiable renderer
  • optimization of Gaussian properties

Models Used

  • Mip-NeRF360
  • InstantNGP
  • Plenoxels

Datasets

The following datasets were used in this research:

  • Mip-NeRF360
  • Tanks&Temples
  • Deep Blending
  • NeRF-synthetic
  • Blender

Evaluation Metrics

  • PSNR
  • L-PIPS
  • SSIM

Results

  • Real-time rendering at 93 fps
  • State-of-the-art quality compared to previous methods with lower training times
  • High visual quality in novel view synthesis from multiple images

Limitations

The authors identified the following limitations:

  • Artifacts in poorly observed areas
  • Popping artifacts in large Gaussians
  • Lack of regularization in optimization

Technical Requirements

  • Number of GPUs: 1
  • GPU Type: A6000

Keywords

3D Gaussian radiance fields real-time rendering neural radiance field view synthesis

Papers Using Similar Methods

External Resources