EasyCom

Dataset Information
Modalities
Images, Videos, Audio, Time series, Speech, Dialog, RGB Video
Languages
English
Introduced
2021
License
Homepage

Overview

The Easy Communications (EasyCom) dataset is a world-first dataset designed to help mitigate the cocktail party effect from an augmented-reality (AR) -motivated multi-sensor egocentric world view. The dataset contains AR glasses egocentric multi-channel microphone array audio, wide field-of-view RGB video, speech source pose, headset microphone audio, annotated voice activity, speech transcriptions, head and face bounding boxes and source identification labels. We have created and are releasing this dataset to facilitate research in multi-modal AR solutions to the cocktail party problem.

Source: EasyCom

Variants: EasyCom

Associated Benchmarks

This dataset is used in 3 benchmarks:

Recent Benchmark Submissions

Task Model Paper Date
Speech Recognition ReVISE (bf) ReVISE: Self-Supervised Speech Resynthesis with … 2022-12-21
Speech Recognition ReVISE (ch2) ReVISE: Self-Supervised Speech Resynthesis with … 2022-12-21
Speech Enhancement ReVISE (ch2) ReVISE: Self-Supervised Speech Resynthesis with … 2022-12-21
Speech Enhancement ReVISE (bf) ReVISE: Self-Supervised Speech Resynthesis with … 2022-12-21
Speech Enhancement Demucs (ch2) ReVISE: Self-Supervised Speech Resynthesis with … 2022-12-21
Speech Enhancement Demucs (bf) ReVISE: Self-Supervised Speech Resynthesis with … 2022-12-21
Speech Recognition Demucs (bf) ReVISE: Self-Supervised Speech Resynthesis with … 2022-12-21
Speech Recognition Demucs (ch2) ReVISE: Self-Supervised Speech Resynthesis with … 2022-12-21
Speech Recognition DAJA (MVDR,HMA,1000) (Overlapped Speech) Direction-Aware Joint Adaptation of Neural … 2022-07-15
Speech Enhancement DAJA (MVDR,HMA,1000) (Overlapped Speech) Direction-Aware Joint Adaptation of Neural … 2022-07-15
Active Speaker Localization AV (cor+eng+box) Egocentric Deep Multi-Channel Audio-Visual Active … 2022-01-06
Speech Enhancement MaxDI (Baseline) EasyCom: An Augmented Reality Dataset … 2021-07-09

Research Papers

Recent papers with results on this dataset: