ReLU
Rectified Linear Unit


従来のS字カーブであるtanhに比べて性能が良いことが下記論文で示された
Glorot, X., Bordes, A. and Bengio, Y., 2011, June. Deep sparse rectifier neural networks. In Proceedings of the fourteenth international conference on artificial intelligence and statistics (pp. 315-323).



Related