Sarah Hoffmann 92f6ec2328 remove support for properties on variants ...

Those are not going to be used in the near future, so no need to
carry that code around just now.

2021-10-05 10:29:36 +02:00

..

sanitizers

add unit tests for new sanatizer functions

2021-10-01 12:27:24 +02:00

token_analysis

remove support for properties on variants

2021-10-05 10:29:36 +02:00

__init__.py

introduce tokenizer modules

2021-04-30 11:29:57 +02:00

base.py

unify ICUNameProcessorRules and ICURuleLoader

2021-10-01 12:27:24 +02:00

factory.py

unify ICUNameProcessorRules and ICURuleLoader

2021-10-01 12:27:24 +02:00

icu_rule_loader.py

move parsing of token analysis config to analyzer

2021-10-04 18:31:58 +02:00

icu_tokenizer.py

introduce sanitizer step before token analysis

2021-10-01 12:27:24 +02:00

legacy_tokenizer.py

unify ICUNameProcessorRules and ICURuleLoader

2021-10-01 12:27:24 +02:00

place_sanitizer.py

introduce sanitizer step before token analysis

2021-10-01 12:27:24 +02:00