Hypernetworks
After many iterations testing many different architectures, Aero was able to come up with one that is both performant and achieves high accuracy with varied dataset sizes. The hypernets are applied to the k and v vectors of CrossAttention layers in StableDiffusion, while not touching any other parts of the U-net.
TL;DR
Stable-Diffusionのチューニング方法
U-NetのCross-Attentionに対して, keyとvalueにのみ新しいNNを適用することで, 既存のパラメタを変えることなくチューニングすることができる