Sarah Hoffmann 9ba2019470 precompute replacements while loading configuration

2021-10-05 10:20:08 +02:00

..

sanitizers

add unit tests for new sanatizer functions

2021-10-01 12:27:24 +02:00

token_analysis

precompute replacements while loading configuration

2021-10-05 10:20:08 +02:00

__init__.py

introduce tokenizer modules

2021-04-30 11:29:57 +02:00

base.py

unify ICUNameProcessorRules and ICURuleLoader

2021-10-01 12:27:24 +02:00

factory.py

unify ICUNameProcessorRules and ICURuleLoader

2021-10-01 12:27:24 +02:00

icu_rule_loader.py

move parsing of token analysis config to analyzer

2021-10-04 18:31:58 +02:00

icu_tokenizer.py

introduce sanitizer step before token analysis

2021-10-01 12:27:24 +02:00

icu_variants.py

unify ICUNameProcessorRules and ICURuleLoader

2021-10-01 12:27:24 +02:00

legacy_tokenizer.py

unify ICUNameProcessorRules and ICURuleLoader

2021-10-01 12:27:24 +02:00

place_sanitizer.py

introduce sanitizer step before token analysis

2021-10-01 12:27:24 +02:00