mirror of
https://github.com/osm-search/Nominatim.git
synced 2026-03-14 23:14:07 +00:00
introduce sanitizer step before token analysis
Sanatizer functions allow to transform name and address tags before they are handed to the tokenizer. Theses transformations are visible only for the tokenizer and thus only have an influence on the search terms and address match terms for a place. Currently two sanitizers are implemented which are responsible for splitting names with multiple values and removing bracket additions. Both was previously hard-coded in the tokenizer.
This commit is contained in:
22
nominatim/tokenizer/sanitizers/strip_brace_terms.py
Normal file
22
nominatim/tokenizer/sanitizers/strip_brace_terms.py
Normal file
@@ -0,0 +1,22 @@
|
||||
"""
|
||||
Sanitizer handling names with addendums in braces.
|
||||
"""
|
||||
|
||||
def create(_):
|
||||
""" Create a name processing function that creates additional name variants
|
||||
when a name has an addendum in brackets (e.g. "Halle (Saale)"). The
|
||||
additional variant only contains the main name without the bracket part.
|
||||
"""
|
||||
def _process(obj):
|
||||
""" Add variants for names that have a bracket extension.
|
||||
"""
|
||||
new_names = []
|
||||
if obj.names:
|
||||
for name in (n for n in obj.names if '(' in n.name):
|
||||
new_name = name.name.split('(')[0].strip()
|
||||
if new_name:
|
||||
new_names.append(name.clone(name=new_name))
|
||||
|
||||
obj.names.extend(new_names)
|
||||
|
||||
return _process
|
||||
Reference in New Issue
Block a user