Group normalization

Author: Yuxin Wu Kaiming He Abstract: Batch Normalization (BN) is a milestone technique in the development of deep learning, enabling various networks to train. However, normalizing along the batch dimension introduces problems—BN’s error increases rapidly when the...

Backprop Evolution

Author(s): Alber, MaximilianBello, IrwanZoph, BarretKindermans, Pieter-JanRamachandran, PrajitLe, Quoc Abstract: The back-propagation algorithm is the cornerstone of deep learning. Despite its importance, few variations of the algorithm have been attempted. This work...
The SELF Institute