SLUE

Spoken Language Understanding Evaluation

Dataset Information
Modalities
Speech
Introduced
2021
License
Custom
Homepage
Official Website
Contents

Overview

Spoken Language Understanding Evaluation (SLUE) is a suite of benchmark tasks for spoken language understanding evaluation. It consists of limited-size labeled training sets and corresponding evaluation sets. This resource would allow the research community to track progress, evaluate pre-trained representations for higher-level tasks, and study open questions such as the utility of pipeline versus end-to-end approaches. The first phase of the SLUE benchmark suite consists of named entity recognition (NER), sentiment analysis (SA), and ASR on the corresponding datasets.

Corpus includes:

Variants: SLUE

Associated Benchmarks

This dataset is used in 3 benchmarks:

Recent Benchmark Submissions

Task Model Paper Date
Named Entity Recognition (NER) Wav2Seq (from HuBERT-large) Wav2Seq: Pre-training Speech-to-Text Encoder-Decoder Models … 2022-05-02
Speech Recognition W2V2-B-LS960 (+ TED-LIUM 3 LM) SLUE: New Benchmark Tasks for … 2021-11-19
Speech Recognition W2V2-L-LL60K (+ in-domain LM) SLUE: New Benchmark Tasks for … 2021-11-19
Speech Recognition W2V2-L-LL60K SLUE: New Benchmark Tasks for … 2021-11-19
Speech Recognition W2V2-B-LS960 (+ in-domain LM) SLUE: New Benchmark Tasks for … 2021-11-19
Speech Recognition W2V2-B-LS960 SLUE: New Benchmark Tasks for … 2021-11-19
Speech Recognition HuBERT-B-LS960 SLUE: New Benchmark Tasks for … 2021-11-19
Speech Recognition W2V2-B-VP100K SLUE: New Benchmark Tasks for … 2021-11-19
Sentiment Analysis W2V2-L-LL60K (pipeline approach, uses LM) SLUE: New Benchmark Tasks for … 2021-11-19
Sentiment Analysis W2V2-L-LL60K (pipeline approach) SLUE: New Benchmark Tasks for … 2021-11-19
Sentiment Analysis W2V2-B-LS960 (pipeline approach, uses LM) SLUE: New Benchmark Tasks for … 2021-11-19
Sentiment Analysis W2V2-B-LS960 (pipeline approach) SLUE: New Benchmark Tasks for … 2021-11-19
Sentiment Analysis W2V2-L-LL60K (e2e approach) SLUE: New Benchmark Tasks for … 2021-11-19
Sentiment Analysis HuBERT-B-LS960 (e2e approach) SLUE: New Benchmark Tasks for … 2021-11-19
Sentiment Analysis W2V2-B-LS960 (e2e approach) SLUE: New Benchmark Tasks for … 2021-11-19
Sentiment Analysis W2V2-B-VP100K (e2e approach) SLUE: New Benchmark Tasks for … 2021-11-19
Named Entity Recognition (NER) W2V2-L-LL60K (pipeline approach, uses LM) SLUE: New Benchmark Tasks for … 2021-11-19
Named Entity Recognition (NER) W2V2-B-LS960 (pipeline approach, uses LM) SLUE: New Benchmark Tasks for … 2021-11-19
Named Entity Recognition (NER) W2V2-L-LL60K (e2e approach, uses LM) SLUE: New Benchmark Tasks for … 2021-11-19
Named Entity Recognition (NER) W2V2-B-LS960 (e2e approach, uses LM) SLUE: New Benchmark Tasks for … 2021-11-19

Research Papers

Recent papers with results on this dataset:

External Links: