datasets.DatasetDict
https://huggingface.co/docs/datasets/main/en/package_reference/main_classes#datasets.DatasetDict
Dictionary with split names as keys (‘train’, ‘test’ for example), and Dataset objects as values.
trainデータはこれ、testデータはこれと辞書で持つ
辞書のvalueは datasets.Dataset
It also has dataset transform methods like map or filter, to process all the splits at once.
map()でtrainもtestもまとめて処理!
https://huggingface.co/docs/datasets/main/en/package_reference/main_classes#datasets.DatasetDict.map
The transformation is applied to all the datasets of the dataset dictionary.
push_to_hub()
private
If None (default), the repo will be public unless the organization’s default is private.
This value is ignored if the repo already exists.
push_to_hubでHugging Faceにリポジトリを作るときの可視性の指定