WiC-TSV

Words-in-Context: Target Sense Verification

Dataset Information
Modalities
Texts
Languages
English
Introduced
2020
License
MIT
Homepage
Official Website
Contents

Overview

WiC-TSV is a new multi-domain evaluation benchmark for Word Sense Disambiguation. More specifically, it is a framework for Target Sense Verification of Words in Context which grounds its uniqueness in the formulation as a binary classification task thus being independent of external sense inventories, and the coverage of various domains. This makes the dataset highly flexible for the evaluation of a diverse set of models and systems in and across domains. WiC-TSV provides three different evaluation settings, depending on the input signals provided to the model.

Variants: WiC-TSV

Associated Benchmarks

This dataset is used in 2 benchmarks:

  • Word Sense Disambiguation - Metrics: Task 1 Accuracy: all, Task 1 Accuracy: general purpose, Task 1 Accuracy: domain specific, Task 2 Accuracy: all, Task 2 Accuracy: general purpose, Task 2 Accuracy: domain specific, Task 3 Accuracy: all, Task 3 Accuracy: general purpose, Task 3 Accuracy: domain specific
  • Entity Linking - Metrics: Task 1 Accuracy: all, Task 1 Accuracy: general purpose, Task 1 Accuracy: domain specific, Task 2 Accuracy: all, Task 2 Accuracy: general purpose, Task 2 Accuracy: domain specific, Task 3 Accuracy: all, Task 3 Accuracy: general purpose, Task 3 Accuracy: domain specific

Recent Benchmark Submissions

Task Model Paper Date
Entity Linking All true WiC-TSV: An Evaluation Benchmark for … 2020-04-30
Entity Linking Human WiC-TSV: An Evaluation Benchmark for … 2020-04-30
Word Sense Disambiguation Unsupervised Bert WiC-TSV: An Evaluation Benchmark for … 2020-04-30
Word Sense Disambiguation FastText WiC-TSV: An Evaluation Benchmark for … 2020-04-30
Word Sense Disambiguation All true WiC-TSV: An Evaluation Benchmark for … 2020-04-30
Word Sense Disambiguation Human WiC-TSV: An Evaluation Benchmark for … 2020-04-30
Word Sense Disambiguation Bert-base WiC-TSV: An Evaluation Benchmark for … 2020-04-30
Entity Linking Bert-base WiC-TSV: An Evaluation Benchmark for … 2020-04-30
Entity Linking Unsupervised Bert WiC-TSV: An Evaluation Benchmark for … 2020-04-30
Entity Linking FastText WiC-TSV: An Evaluation Benchmark for … 2020-04-30
Entity Linking GlossBert-ws GlossBERT: BERT for Word Sense … 2019-08-20
Word Sense Disambiguation GlossBert-ws GlossBERT: BERT for Word Sense … 2019-08-20

Research Papers

Recent papers with results on this dataset:

External Links: