rouge-score
https://pypi.org/project/rouge-score/
This is a native python implementation of ROUGE, designed to replicate results from the original perl package.
#ROUGE
https://github.com/google-research/google-research/tree/master/rouge
RougeScorer https://github.com/google-research/google-research/blob/bbb2376225c54168e44c2f749aa1173ee0ed5991/rouge/rouge_scorer.py#L50
tokenizer: Tokenizer object which has a tokenize() method.
デフォルト値はNone -> DefaultTokenizer
👉rouge_score.rouge_scorer.RougeScorer
DefaultTokenizer https://github.com/google-research/google-research/blob/bbb2376225c54168e44c2f749aa1173ee0ed5991/rouge/tokenizers.py#L38
Default tokenizer which tokenizes on whitespace.
stemmer=Noneでtokenize.tokenizeを読み込む
tokenize関数 https://github.com/google-research/google-research/blob/bbb2376225c54168e44c2f749aa1173ee0ed5991/rouge/tokenize.py#L35
Replace any non-alpha-numeric characters with spaces.
日本語は除かれる