Randomized AMRs with Rewired Edges
RARE consists of English AMR pairs with similarity scores that reflect the structural differences between them.
Given that AMRs are graphical representations of text, an AMR similarity metric should be sensitive to structural variations between AMRs, even if its labels remain unchanged. Since there is no established evaluation of AMR metrics on structural similarity, we have developed a new benchmark dataset called Randomized AMRs with Rewired Edges (RARE). RARE consists of English AMR pairs with similarity scores that reflect the structural differences between them.
It can be used to evaluate AMR metrics on their sensitivity at detecting structural changes between pairs of AMRs.
Variants: RARE
This dataset is used in 1 benchmark:
| Task | Model | Paper | Date |
|---|---|---|---|
| Graph Matching | Rematch | Rematch: Robust and Efficient Matching … | 2024-04-02 |
| Graph Matching | WLK | Weisfeiler-Leman in the BAMBOO: Novel … | 2021-08-26 |
| Graph Matching | S2match | AMR Similarity Metrics from Principles | 2020-01-29 |
| Graph Matching | SemBleu | SemBleu: A Robust Metric for … | 2019-05-26 |
Recent papers with results on this dataset: