Sarah Hoffmann 872ab91421 fix name of transliterator ...

Should be different from the normalisation rules.

2021-05-05 17:09:38 +02:00

..

__init__.py

introduce tokenizer modules

2021-04-30 11:29:57 +02:00

factory.py

boilerplate for PHP code of tokenizer

2021-04-30 11:31:52 +02:00

legacy_icu_tokenizer.py

fix name of transliterator

2021-05-05 17:09:38 +02:00

legacy_tokenizer.py

enable BDD tests for different tokenizers

2021-05-05 10:31:51 +02:00