4ab46088eec4,053
http://nhiro.org.s3.amazonaws.com/3/5/3506ba7f9411da9f94f0fc7f54901e7a.jpg https://gyazo.com/3506ba7f9411da9f94f0fc7f54901e7a
4ab46088eec4,052 4ab46088eec4,054
4ab46088eec4
(OCR text)
54
maskLM
Transformer実装からmaskを全部取り除いて
"0-9の繰り返し"言語において穴埋めができた
epoch 0, loss 0.2409, time-9.0
Q:8901234+67? A:8901234267 T:8901234567 NG
epoch 1, loss=0.6295, time=18.0
Q:456789012+? A:4567890128 T:4567890123 NG
epoch 2, loss-0.4144, time-27.2
Q:23+5678901? A:2365678901 T:2345678901 NG
epoch 3, loss 0.1744, time=36.4
Q: 9012345+78? A:9012345678 T:9012345678 OK
epoch 4, loss-0.1293, time=45.5
Q:45678+0123? A:4567890123 T:4567890123 OK
epoch 5, loss-0.1969, time-54.5
Q:0123+56789? A:0123456789 T:0123456789 OK
epoch 6, loss 0.00 10, time=63.7
Q:+234567890? A:1234567890 T:1234567890 OK
epoch 7, loss-0.1407, time=72.9
Q:678901234+? A:6789012345 T:6789012345 OK
"+"がMASK
II
Nearest: b30136bf0f8c,021