nuScenes

Overview edit

Variants: nuScenes, nuScenes Camera Only, nuscenes Camera-Radar

Associated Benchmarks

This dataset is used in 14 benchmarks:

• Trajectory Prediction - Metrics: MinADE_5, MinADE_10, MissRateTopK_2_5, MissRateTopK_2_10, MinFDE_1, OffRoadRate
• Prediction Of Occupancy Grid Maps - Metrics: mIoU
• Motion Detection - Metrics: F1 (%)
• Object Detection - Metrics: AP(l), AP(m), AP(s), AP50, AP75, AP85, AR, AR(l), AR(m), AR(s), MAP
• Instance Segmentation - Metrics: MOTA
• 3D Semantic Segmentation - Metrics: mIoU
• Motion Planning - Metrics: Collision, L2
• 3D Object Detection - Metrics: NDS, mAP, mATE, mASE, mAOE, mAVE, mAAE
• 3D Multi-Object Tracking - Metrics: AMOTA, MOTA, Recall
• LIDAR Semantic Segmentation - Metrics: test mIoU, val mIoU
• Lane Detection - Metrics: IoU, F1 score
• HD semantic map learning - Metrics: Chamfer AP
• Trajectory Planning - Metrics: Collision-3s, L2-3s, Collision-1s, Collision-2s, Collision-Avg, L2-1s, L2-2s, L2-Avg
• Bird's-Eye View Semantic Segmentation - Metrics: IoU veh - 224x480 - Vis filter. - 100x100 at 0.5, IoU veh - 448x800 - Vis filter. - 100x100 at 0.5, IoU veh - 224x480 - No vis filter - 100x100 at 0.5, IoU veh - 448x800 - No vis filter - 100x100 at 0.5, IoU ped - 224x480 - Vis filter. - 100x100 at 0.5, IoU lane - 224x480 - 100x100 at 0.5, IoU veh - 224x480 - No vis filter - 100x50 at 0.25, IoU vehicle - Setting 3

Recent Benchmark Submissions

Research Papers

Recent papers with results on this dataset:

• DINO in the Room: Leveraging 2D Foundation Models for 3D Segmentation (2025) -
• SpaRC: Sparse Radar-Camera Fusion for 3D Object Detection (2024) -
• MCTrack: A Unified 3D Multi-Object Tracking Framework for Autonomous Driving (2024) -
• Robust Multimodal 3D Object Detection via Modality-Agnostic Decoding and Proximity-based Modality Ensemble (2024) -
• Serialized Point Mamba: A Serialized Point Cloud Mamba Segmentation Model (2024) -
• SFPNet: Sparse Focal Point Network for Semantic Segmentation on General LiDAR Point Clouds (2024) -
• SemanticFormer: Holistic and Semantic Traffic Scene Representation for Trajectory Prediction using Knowledge Graphs (2024) -
• LSK3DNet: Towards Effective and Efficient 3D Perception with Large Sparse Kernels (2024) -
• UniTraj: A Unified Framework for Scalable Vehicle Trajectory Prediction (2024) -
• OA-CNNs: Omni-Adaptive Sparse CNNs for 3D Semantic Segmentation (2024) -
• Fast-Poly: A Fast Polyhedral Framework For 3D Multi-Object Tracking (2024) -
• Unleashing HyDRa: Hybrid Fusion, Depth Consistency and Radar for Unified 3D Perception (2024) -
• Point Transformer V3: Simpler, Faster, Stronger (2023) -
• FRNet: Frustum-Range Networks for Scalable LiDAR Segmentation (2023) -
• PointBeV: A Sparse Approach to BeV Predictions (2023) -
• GPT-Driver: Learning to Drive with GPT (2023) -
• Offline Tracking with Object Permanence (2023) -
• Towards Large-scale 3D Representation Learning with Multi-dataset Point Prompt Training (2023) -
• CASPNet++: Joint Multi-Agent Motion Prediction (2023) -
• UniTR: A Unified and Efficient Multi-Modal Transformer for Bird's-Eye-View Representation (2023) -