Sarah Hoffmann 7e7dd769fd remove language and partition from name import

2021-09-02 14:41:11 +02:00

..

__init__.py

introduce tokenizer modules

2021-04-30 11:29:57 +02:00

base.py

define formal public Python interface for tokenizer

2021-08-16 11:41:54 +02:00

factory.py

do not hide errors when importing tokenizer

2021-05-18 16:28:21 +02:00

icu_name_processor.py

split up variant computation for better readability

2021-07-12 14:43:50 +02:00

icu_rule_loader.py

remove language and partition from name import

2021-09-02 14:41:11 +02:00

icu_tokenizer.py

rename legacy_icu tokenizer to icu tokenizer

2021-08-17 23:11:47 +02:00

icu_variants.py

remove default parameter for namedtuple

2021-07-06 22:57:42 +02:00

legacy_tokenizer.py

define formal public Python interface for tokenizer

2021-08-16 11:41:54 +02:00