Transformer
https://gyazo.com/76f4f83f061c152b478c35ea07cd4cf7
左 encoder
翻訳元の文章を理解するのが得意な構造
You'll find the key repository boundaries in this illustration: a Transformer is generally made of a collection of Attention mechanism, embeddings to encode some positional information, feed-forward blocks and a residual path (typically referred to as pre- or post- layer norm). Transformerは本文がわかりづらい
わかるように全部書いてやるという論文
アーキテクチャの基本形は3つのパターン
decoder / encoder / trasnformer
encder / transformer
decoder / transfomer
GPTはこれ
decoderのような単語は歴史的経緯で、別にdecodeしてない
https://www.youtube.com/watch?v=50XvMaWhiTY&list=PLhDAH9aTfnxKXf__soUoAEOrbLAOnVHCP&index=29
https://www.youtube.com/watch?v=FFoLqib6u-0&list=PLhDAH9aTfnxKXf__soUoAEOrbLAOnVHCP&index=30
https://www.youtube.com/watch?v=n1QYofU3_hY&list=PLhDAH9aTfnxKXf__soUoAEOrbLAOnVHCP&index=40