← ML Research Wiki / 1608.06993

Densely Connected Convolutional Networks

Gao Huang Facebook AI Research Cornell University Tsinghua University Cornell University, Zhuang Liu [email protected] Facebook AI Research Cornell University Tsinghua University Cornell University, Laurens Van Der Maaten Facebook AI Research Cornell University Tsinghua University Cornell University, Kilian Q Weinberger Facebook AI Research Cornell University Tsinghua University Cornell University (2016)

Paper Information
arXiv ID
1608.06993
Venue
Computer Vision and Pattern Recognition
Domain
Not specified
SOTA Claim
Yes
Contents

Abstract

Recent work has shown that convolutional networks can be substantially deeper, more accurate, and efficient to train if they contain shorter connections between layers close to the input and those close to the output. In this paper, we embrace this observation and introduce the Dense Convolutional Network (DenseNet), which connects each layer to every other layer in a feed-forward fashion. Whereas traditional convolutional networks with L layers have L connections-one between each layer and its subsequent layer-our network has L(L+1) 2 direct connections. For each layer, the feature-maps of all preceding layers are used as inputs, and its own feature-maps are used as inputs into all subsequent layers. DenseNets have several compelling advantages: they alleviate the vanishing-gradient problem, strengthen feature propagation, encourage feature reuse, and substantially reduce the number of parameters. We evaluate our proposed architecture on four highly competitive object recognition benchmark tasks (CIFAR-10, CIFAR-100, SVHN, and ImageNet). DenseNets obtain significant improvements over the state-of-the-art on most of them, whilst requiring less computation to achieve high performance. Code and pre-trained models are available at

Summary

The paper introduces Dense Convolutional Networks (DenseNets), a novel architecture designed to enhance convolutional neural networks by establishing dense connectivity among layers. Each layer is connected to every preceding layer, allowing for improved feature propagation and reuse, while addressing issues like the vanishing gradient problem. DenseNets are evaluated on multiple challenging object recognition datasets, including CIFAR-10, CIFAR-100, SVHN, and ImageNet, where they demonstrate superior performance compared to traditional architectures, particularly in terms of efficiency and accuracy. The paper discusses the advantages of its architecture, such as reduced parameters, better training dynamics, and regularization effects, ultimately achieving state-of-the-art results across these benchmarks. DenseNets simplify deep networks by allowing feature reuse without requiring extensive redundancy, making them competitive and efficient models for visual recognition tasks.

Methods

This paper employs the following methods:

  • DenseNet
  • Convolutional Neural Networks

Models Used

  • DenseNet-BC

Datasets

The following datasets were used in this research:

  • CIFAR-10
  • CIFAR-100
  • SVHN
  • ImageNet

Evaluation Metrics

  • Error Rate
  • Accuracy

Results

  • DenseNets achieve significant improvements over state-of-the-art models on CIFAR-10, CIFAR-100, SVHN, and ImageNet.
  • Outperform existing algorithms with fewer parameters.

Limitations

The authors identified the following limitations:

  • Not specified

Technical Requirements

  • Number of GPUs: None specified
  • GPU Type: None specified

Papers Using Similar Methods

External Resources

Funding: Not specified
References: 51
Influential Citations: 4450