SNIPS

SNIPS Natural Language Understanding benchmark

Dataset Information
Modalities
Texts
Languages
Spanish
Introduced
2018
License
Unknown
Homepage

Overview

The SNIPS Natural Language Understanding benchmark is a dataset of over 16,000 crowdsourced queries distributed among 7 user intents of various complexity:

  • SearchCreativeWork (e.g. Find me the I, Robot television show),
  • GetWeather (e.g. Is it windy in Boston, MA right now?),
  • BookRestaurant (e.g. I want to book a highly rated restaurant in Paris tomorrow night),
  • PlayMusic (e.g. Play the last track from Beyoncé off Spotify),
  • AddToPlaylist (e.g. Add Diamonds to my roadtrip playlist),
  • RateBook (e.g. Give 6 stars to Of Mice and Men),
  • SearchScreeningEvent (e.g. Check the showtimes for Wonder Woman in Paris).
    The training set contains of 13,084 utterances, the validation set and the test set contain 700 utterances each, with 100 queries per intent.

Source: https://paperswithcode.com/paper/snips-voice-platform-an-embedded-spoken/

Variants: SNIPS

Associated Benchmarks

This dataset is used in 5 benchmarks:

Recent Benchmark Submissions

Task Model Paper Date
Slot Filling CTRAN CTRAN: CNN-Transformer-based Network for Natural … 2023-03-19
Intent Detection CTRAN CTRAN: CNN-Transformer-based Network for Natural … 2023-03-19
Intent Discovery k-PCA + HDBSCAN A Hybrid Architecture for Out … 2023-03-07
Intent Detection LIDSNet LIDSNet: A Lightweight on-device Intent … 2021-10-06
Zero-Shot Learning ZSL-KG Zero-Shot Learning with Common Sense … 2020-06-18
Slot Filling AGIF AGIF: An Adaptive Graph-Interactive Framework … 2020-04-21
Intent Detection AGIF AGIF: An Adaptive Graph-Interactive Framework … 2020-04-21
Open Intent Discovery CDAC+ Discovering New Intents via Constrained … 2019-11-20
Slot Filling Context Encoder Improving Slot Filling by Utilizing … 2019-11-05
Slot Filling Stack-Propagation (+BERT) A Stack-Propagation Framework with Token-Level … 2019-09-05
Intent Detection Stack-Propagation (+BERT) A Stack-Propagation Framework with Token-Level … 2019-09-05
Slot Filling Stack-Propagation A Stack-Propagation Framework with Token-Level … 2019-09-05
Intent Detection Stack-Propagation A Stack-Propagation Framework with Token-Level … 2019-09-05
Intent Detection SF-ID A Novel Bi-directional Interrelated Model … 2019-06-30
Slot Filling SF-ID A Novel Bi-directional Interrelated Model … 2019-06-30
Intent Detection SF-ID (BLSTM) network A Novel Bi-directional Interrelated Model … 2019-06-30
Intent Detection Capsule-NLU Joint Slot Filling and Intent … 2018-12-22
Slot Filling Capsule-NLU Joint Slot Filling and Intent … 2018-12-22

Research Papers

Recent papers with results on this dataset: