9fc872a2a6e4028

http://nhiro.org.s3.amazonaws.com/9/e/9e5fc59ccfa402f6846e49e058e78a64.jpg https://gyazo.com/9e5fc59ccfa402f6846e49e058e78a64

(OCR text)

実験

29

Adam TAdaDelta Nesterovb j

CIFAR10 ConvNet

MNIST Multilayer Neural Network+ dropout

10

AdaGrad

102

AdaGrad

AdaGrad+dropout

RMSProp

SGDNesterov

SGDNesterov

SGDNesterov+dropout

10

AdaDelta

Adam

Adam+dropout

Adam

10°

10

102

102

103

10

10

15

20

25

iterations over entire dataset

30

35

40

100

50

150

iterations over entire dataset

200

training cost

45

training cost