处理multi-language model的数据size不平衡的问题

在NLP中，处理multi-language 是相对棘手的问题，尤其是在语言种类较多的情况比如几十种不同的语言。因为在multi-language model 中训练数据可能不平衡，包括数据丰富和贫乏的不同数据，在这种情况下的数据集可以使用一个shared vocabulary。尽管不同语言中的相同单词可以含义不同，但如果对每种语言都创建一个单词库（vocabulary），计算会更加耗时。

In NLP, dealing with multi language is a relatively tricky problem, especially when there are many kinds of languages, like dozens of different languages. Because the training data in the multi-language model may be unbalanced, including high-resource and low-resourc.
In this case, we can use a shared vocabulary.Although same word can have different meaning in different language. But there is a small number for low-resource languages. If we create vocabularies for each language, the computational cost will be expensive.