2 Commits

Author	SHA1	Message	Date
Sarah Hoffmann	b7d4ff3201	icu: normalise simplified to traditional chinese ... The conversion is unambigious in most cases, so that the information loss is minimal.	2021-08-31 11:18:34 +02:00
Sarah Hoffmann	118858a55e	rename legacy_icu tokenizer to icu tokenizer ... The new icu tokenizer is now no longer compatible with the old legacy tokenizer in terms of data structures. Therefore there is also no longer a need to refer to the legacy tokenizer in the name.	2021-08-17 23:11:47 +02:00