ReLU
Rectified Linear Unit
https://gyazo.com/848e2236bab02732e1979df7acdf684c
従来のS字カーブであるtanhに比べて性能が良いことが下記論文で示された
Glorot, X., Bordes, A. and Bengio, Y., 2011, June. Deep sparse rectifier neural networks. In Proceedings of the fourteenth international conference on artificial intelligence and statistics (pp. 315-323).
http://proceedings.mlr.press/v15/glorot11a/glorot11a.pdf
https://gyazo.com/049c141201bc18c684db3943db8a2c43
https://gyazo.com/febe2e044d1c5b503e7f66a8ffa68bb5