ignore Unicode format characters for normalization

Also adds tests. Fixes #1007.
2018-04-10 22:48:17 +02:00
parent 28ee59dd64
commit ae83ceab5e
5 changed files with 22 additions and 2 deletions
--- a/test/bdd/db/query/normalization.feature
+++ b/test/bdd/db/query/normalization.feature
@@ -136,3 +136,13 @@ Feature: Import and search of names
        Then results contain
         | ID | osm_type | osm_id |
         | 0  | R        | 1 |
+
+     Scenario: Unprintable characters in postcodes are ignored
+        Given the named places
+            | osm  | class   | type   | address |
+            | N234 | amenity | prison | 'postcode' : u'1234\u200e' |
+        When importing
+        And searching for "1234"
+        Then results contain
+         | ID | osm_type |
+         | 0  | P        |