rename legacy_icu tokenizer to icu tokenizer

The new icu tokenizer is now no longer compatible with the old
legacy tokenizer in terms of data structures. Therefore there
is also no longer a need to refer to the legacy tokenizer in the
name.
This commit is contained in:
Sarah Hoffmann
2021-08-17 23:11:47 +02:00
parent 656c1291b1
commit 118858a55e
10 changed files with 22 additions and 16 deletions