add japanese sanitizer

2026-02-16 15:47:58 +00:00 · 2023-07-26 07:54:58 +00:00
parent 261e0cfd5a
commit 0722495434
5 changed files with 253 additions and 0 deletions
--- a/docs/customize/Tokenizers.md
+++ b/docs/customize/Tokenizers.md
@@ -229,6 +229,14 @@ The following is a list of sanitizers that are shipped with Nominatim.
    rendering:
        heading_level: 6

+#### tag-japanese
+
+::: nominatim.tokenizer.sanitizers.tag_japanese
+    selection:
+        members: False
+    rendering:
+        heading_level: 6
+
 #### Token Analysis

 Token analyzers take a full name and transform it into one or more normalized