英語のfill-maskの例
モデルを指定しない場合、distilroberta-baseが使われる
code:en_fill_mask.py
>> from transformers import pipeline
>> nlp = pipeline("fill-mask")
>> nlp.tokenizer.mask_token
'<mask>'
>> results = nlp(f"I eat {nlp.tokenizer.mask_token} everyday") # <mask>を入れたテキストを作る（そこを埋めさせる）
>> for result in results: # scoreってあんまり意味がない？
... print(result)
...
{'score': 0.07874908298254013, 'token': 9366, 'token_str': ' pizza', 'sequence': 'I eat pizza everyday'}
{'score': 0.03790418803691864, 'token': 33678, 'token_str': ' peanuts', 'sequence': 'I eat peanuts everyday'}
{'score': 0.03601660206913948, 'token': 18236, 'token_str': ' pasta', 'sequence': 'I eat pasta everyday'}
{'score': 0.03574463352560997, 'token': 28287, 'token_str': ' sushi', 'sequence': 'I eat sushi everyday'}
{'score': 0.03567703440785408, 'token': 24351, 'token_str': ' yogurt', 'sequence': 'I eat yogurt everyday'}