Improved Techniques for Training GANs

https://gyazo.com/d1d2456f5324365f5dc87b520ffecbb6

Feature matching addresses the instability of GANs by specifying a new objective for the generator that prevents it from overtraining on the current discriminator.

Instead of directly maximizing the output of the discriminator, the new objective requires the generator to generate data that matches the statistics of the real data, where we use the discriminator only to specify the statistics that we think are worth matching.

Specifically, we train the generator to match the expected value of the features on an intermediate layer of the discriminator. This is a natural choice of statistics for the generator to match, since by training the discriminator we ask it to find those features that are most discriminative of real data versus data generated by the current model.

というわけで識別器の正解率をあげるのではなく、実データを入れた時の識別器の中間層の値と、生成データを入れた時の識別層の中間層の値がなるべく遠ざかるように識別器を学習する。

Mode Collapsingを避ける

Mode CollapsingとはGANでよくある「MNISTで学習させたら1しか生成しなくなったぞ」

ノイズ源zを無視して常に1を生成したとしてもDはおかしいとは思わないので、そっちの方が楽じゃんとGが思っちゃう

そこでMinibatchのなかのばらつき具合を肯定的に評価するような目的関数にする