Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
著者
Abstract
Deep learning tools have gained tremendous at-tention in applied machine learning. However such tools for regression and classification do not capture model uncertainty. In compari-son, Bayesian models offer a mathematically grounded framework to reason about model un-certainty, but usually come with a prohibitive computational cost. In this paper we develop a new theoretical framework casting dropout train-ing in deep neural networks (NNs) as approxi-mate Bayesian inference in deep Gaussian processes. A direct result of this theory gives us tools to model uncertainty with dropout NNs – extracting information from existing models that has been thrown away so far. This mitigates the problem of representing uncertainty in deep learning without sacrificing either computational complexity or test accuracy. We perform an ex-tensive study of the properties of dropout’s un-certainty. Various network architectures and non-linearities are assessed on tasks of regression and classification, using MNIST as an example. We show a considerable improvement in predic-tive log-likelihood and RMSE compared to ex-isting state-of-the-art methods, and finish by us-ing dropout’s uncertainty in deep reinforcement learning メモ
やっていること
ニューラルネットワークにおいてdropoutをつかうことで、Gaussian processな確率モデルとして知られるベイズ的な近似ができることを示す
理論的に関係を示す
deep learningで不確実性を表現するツールの開発