hans/Nominatim
1
0
Fork 1
mirror of https://github.com/osm-search/Nominatim.git synced 2026-02-16 15:47:58 +00:00
Code Issues Packages Projects Releases Wiki Activity

add documentation for custom token analysis

Browse Source
This commit is contained in:
Sarah Hoffmann
2022-07-29 09:41:28 +02:00
parent 3746befd88
commit f0d640961a
2 changed files with 43 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

22
docs/develop/ICU-Tokenizer-Modules.md
View File

@@ -52,7 +52,8 @@ the function.
### The sanitation function
The sanitation function receives a single object with three members:
The sanitation function receives a single object of type `ProcessInfo`
which has with three members:
* `place`: read-only information about the place being processed.
See PlaceInfo below.
@@ -62,7 +63,7 @@ The sanitation function receives a single object with three members:
is a PlaceName object.
While the `place` member is provided for information only, the `names` and
`address` lists are meant to be manipulated by the sanitizer. If may add and
`address` lists are meant to be manipulated by the sanitizer. It may add and
remove entries, change information within a single entry (for example by
adding extra attributes) or completely replace the list with a different one.
@@ -80,3 +81,20 @@ adding extra attributes) or completely replace the list with a different one.
rendering:
show_source: no
heading_level: 6
## Custom token analysis module
Setup of a token analyser is split into two parts: configuration and
analyser factory. A token analysis module must therefore implement two
functions:
::: nominatim.tokenizer.token_analysis.base.AnalysisModule
rendering:
show_source: no
heading_level: 6
::: nominatim.tokenizer.token_analysis.base.Analyser
rendering:
show_source: no
heading_level: 6