Recurrent Neural Networks

Author(s): MacKay, MatthewVicol, PaulBa, JimmyGrosse, Roger Abstract: Recurrent neural networks (RNNs) provide state-of-the-art performance in processing sequential data but are memory intensive to train, limiting the flexibility of RNN models which can be trained....

Group normalization

Author: Yuxin Wu Kaiming He Abstract: Batch Normalization (BN) is a milestone technique in the development of deep learning, enabling various networks to train. However, normalizing along the batch dimension introduces problems—BN’s error increases rapidly when the...
The SELF Institute