2 Commits

Author	SHA1	Message	Date
Sarah Hoffmann	837d44391c	move generation of normalized token form to analyzer ... This gives the analyzer more flexibility in choosing the normalized form. In particular, an analyzer creating different variants can choose the variant that will be used as the canonical form.	2022-03-01 09:34:32 +01:00
Sarah Hoffmann	b453b0ea95	introduce mutation variants to generic token analyser ... Mutations are regular-expression-based replacements that are applied after variants have been computed. They are meant to be used for variations on character level. Add spelling variations for German umlauts.	2022-01-18 11:09:21 +01:00