← ML Research Wiki / 1710.10903

GRAPH ATTENTION NETWORKS

Petar Veličković [email protected] Department of Computer Science and Technology Montréal Institute for Learning Algorithms Department of Computer Science and Technology Montréal Institu, Guillem Cucurull [email protected] Department of Computer Science and Technology Montréal Institute for Learning Algorithms Department of Computer Science and Technology Montréal Institute for Lear, Arantxa Casanova [email protected] Department of Computer Science and Technology Montréal Institute for Learning Algorithms Department of Computer Science and Technology Montréal Institute for , Adriana Romero [email protected] Department of Computer Science and Technology Montréal Institute for Learning Algorithms Department of Computer Science and Technology Montréal Inst, Pietro Liò [email protected] Department of Computer Science and Technology Montréal Institute for Learning Algorithms Department of Computer Science and Technology Montréal Institute for Learn, Yoshua Bengio Department of Computer Science and Technology Montréal Institute for Learning Algorithms Department of Computer Science and Technology Montréal Institute for Learning Algorithms Centre (2017)

Paper Information
arXiv ID
1710.10903
Venue
International Conference on Learning Representations
Domain
Machine Learning, Graph Neural Networks
SOTA Claim
Yes
Reproducibility
6/10
Contents

Abstract

We present graph attention networks (GATs), novel neural network architectures that operate on graph-structured data, leveraging masked self-attentional layers to address the shortcomings of prior methods based on graph convolutions or their approximations. By stacking layers in which nodes are able to attend over their neighborhoods' features, we enable (implicitly) specifying different weights to different nodes in a neighborhood, without requiring any kind of costly matrix operation (such as inversion) or depending on knowing the graph structure upfront. In this way, we address several key challenges of spectral-based graph neural networks simultaneously, and make our model readily applicable to inductive as well as transductive problems. Our GAT models have achieved or matched state-of-theart results across four established transductive and inductive graph benchmarks: the Cora, Citeseer and Pubmed citation network datasets, as well as a proteinprotein interaction dataset (wherein test graphs remain unseen during training).

Summary

This paper introduces Graph Attention Networks (GATs), novel neural network architectures designed for graph-structured data. The core innovation is leveraging masked self-attention layers to overcome limitations of earlier graph convolutional approaches. GATs allow nodes to attend selectively to their neighborhoods, assigning different weights without costly matrix operations or prior knowledge of the graph structure. The authors validate their model on four established datasets: Cora, Citeseer, Pubmed, and a protein-protein interaction (PPI) dataset, achieving state-of-the-art results in both transductive and inductive tasks. Key features of GATs include high computational efficiency, applicable to graphs of different structures, and enhanced interpretability through learned attention weights.

Methods

This paper employs the following methods:

  • Graph Attention Networks
  • Masked Self-Attention
  • Multi-Head Attention

Models Used

  • GAT

Datasets

The following datasets were used in this research:

  • Cora
  • Citeseer
  • Pubmed
  • PPI

Evaluation Metrics

  • Accuracy
  • Micro-F1

Results

  • State-of-the-art results on Cora, Citeseer, Pubmed and PPI datasets.

Limitations

The authors identified the following limitations:

  • Not specified

Technical Requirements

  • Number of GPUs: None specified
  • GPU Type: None specified

Keywords

graph neural networks self-attention attention mechanisms node classification inductive learning transductive learning

Papers Using Similar Methods

External Resources

Funding: Not specified
References: 45
Influential Citations: 3150