Sarah Hoffmann 243725aae1 icu: move housenumber token computation out of TokenInfo ...

This was the last function to use the cache. There is a more clean
separation of responsibility now.

2022-03-01 09:34:32 +01:00

..

sanitizers

add tests for get_string_list()

2022-02-07 11:22:24 +01:00

token_analysis

move generation of normalized token form to analyzer

2022-03-01 09:34:32 +01:00

__init__.py

introduce tokenizer modules

2021-04-30 11:29:57 +02:00

base.py

add new command for cleaning word tokens

2022-01-20 20:05:15 +01:00

factory.py

add consistent SPDX copyright headers

2022-01-03 16:23:58 +01:00

icu_rule_loader.py

sanitizer: move helpers into a configuration class

2022-02-07 10:48:00 +01:00

icu_token_analysis.py

handle unknown analyzer

2022-03-01 09:34:32 +01:00

icu_tokenizer.py

icu: move housenumber token computation out of TokenInfo

2022-03-01 09:34:32 +01:00

legacy_tokenizer.py

add new command for cleaning word tokens

2022-01-20 20:05:15 +01:00

place_sanitizer.py

sanitizer: move helpers into a configuration class

2022-02-07 10:48:00 +01:00