hans/Nominatim
1
0
Fork 1
mirror of https://github.com/osm-search/Nominatim.git synced 2026-03-09 03:24:06 +00:00
Code Issues Packages Projects Releases Wiki Activity

avoid special characters in word tokens

Browse Source 
Transliteration should only consist of ASCII letters
and numbers. Avoid any other characters.
This commit is contained in:
Sarah Hoffmann
2021-11-10 17:14:13 +01:00
parent 7326b246b7
commit 1886952666
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
settings/icu_tokenizer.yaml
View File

@@ -21,8 +21,8 @@ transliteration:
- !include icu-rules/extended-unicode-to-asccii.yaml - !include icu-rules/extended-unicode-to-asccii.yaml
- ":: Ascii ()" - ":: Ascii ()"
- ":: NFD ()" - ":: NFD ()"
- "[^[:Ascii:]] >"
- ":: lower ()" - ":: lower ()"
- "[^a-z0-9[:Space:]] >"
- ":: NFC ()" - ":: NFC ()"
sanitizers: sanitizers:
- step: split-name-list - step: split-name-list