Hugging Peers

Identity Mappings in Deep Residual Networks

☆ Save
Kaiming He,Xiangyu Zhang,Shaoqing Ren,Jian Sun
European Conference on Computer Vision cs.CVcs.LG
11.5K citations published

Abstract

Deep residual networks have emerged as a family of extremely deep architectures showing compelling accuracy and nice convergence behaviors. In this paper, we analyze the propagation formulations behind the residual building blocks, which suggest that the forward and backward signals can be directly propagated from one block to any other block, when using identity mappings as the skip connections and after-addition activation. A series of ablation experiments support the importance of these identity mappings. This motivates us to propose a new residual unit, which makes training easier and improves generalization. We report improved results using a 1001-layer ResNet on CIFAR-10 (4.62% error) and CIFAR-100, and a 200-layer ResNet on ImageNet. Code is available at: https://github.com/KaimingHe/resnet-1k-layers

Models from this paper 5

facebook esm1b_t33_650M_UR50S
ModelScope
fill-mask pytorch mit
350 downloads 0 likes
birder-project resnet_v2_50_inat21
Hugging Face
image-classification birder apache-2.0
95 downloads 0 likes
timm resnetv2_152x2_bit.goog_in21k
Hugging Face
image-classification timm apache-2.0
85 downloads 1 likes
timmmodel resnetv2_50x1_bit.goog_distilled_in1k
ModelScope
apache-2.0
17 downloads 0 likes
timmmodel resnetv2_101x1_bit.goog_in21k_ft_in1k
ModelScope
apache-2.0
5 downloads 0 likes

Cite this paper

@article{he2016,
  title = {Identity Mappings in Deep Residual Networks},
  author = {Kaiming He and Xiangyu Zhang and Shaoqing Ren and Jian Sun},
  journal = {European Conference on Computer Vision},
  year = {2016},
  month = {Mar},
  doi = {10.48550/arxiv.1603.05027},
  url = {https://doi.org/10.48550/arxiv.1603.05027},
}