Commit Graph

4308 Commits

Author SHA1 Message Date
Sarah Hoffmann
4a00a3c0f5 penalize name token splitting when phrases are used 2023-07-17 16:27:25 +02:00
Sarah Hoffmann
8366e4ca83 penalize search with frequent partials
Avoid search against frequent partials if we have already looked for
the full name equivalents.
2023-07-17 16:27:25 +02:00
Sarah Hoffmann
283db76e45 avoid splitting of first token when a housenumber is present
This only covers the case of <poi name> <street name> <housenumber>
which is exceedingly rare.
2023-07-17 16:27:25 +02:00
Sarah Hoffmann
8a36ed4f6f increase threshold for full name searches
They still should be preferrred over expensive partial name searches.
2023-07-17 16:27:25 +02:00
Sarah Hoffmann
d0f45155c8 fix search for housenumber names
The search still included a lookup of housenumbers in children which is
wrong.
2023-07-17 16:27:25 +02:00
Sarah Hoffmann
2e56182a7f Merge pull request #3110 from lonvia/sql-lambda-queries
Use SQLAlchemy's lambda statements
2023-07-14 18:57:31 +02:00
Sarah Hoffmann
7932b1849b selected lambdas for search 2023-07-14 15:43:29 +02:00
Sarah Hoffmann
886374d779 add lambdas for layer filters 2023-07-14 15:27:20 +02:00
Sarah Hoffmann
d42e2e391f avoid forwarding variables via SQL 2023-07-14 15:27:16 +02:00
Sarah Hoffmann
f264eaeda2 make SQL statements in reverse lambda functions
Further improves internal caching of statements by SQLAlchemy.
2023-07-14 14:39:14 +02:00
Sarah Hoffmann
35fd74af6d Merge pull request #3096 from alfmarcua/search-within-countries-parameter
Parameterise the search only within countries
2023-07-12 17:45:55 +02:00
alfmarcua
4b53cf1464 Split lookupInCountry in two functions and document NOMINATIM_SEARCH_WITHIN_COUNTRIES parameter 2023-07-12 13:53:23 +02:00
alfmarcua
26e78efbb9 Parameterise the search only within countries 2023-07-12 13:53:23 +02:00
Sarah Hoffmann
157f0b8a83 Merge pull request #3109 from lonvia/prepared-statements
Make prepared statements work with Python API
2023-07-10 11:45:29 +02:00
Sarah Hoffmann
d743cf308e avoid index-use on rank parameters for reverse lookups 2023-07-09 22:49:31 +02:00
Sarah Hoffmann
f4cdcb995c force a fixed pool size and make it configurable 2023-07-09 00:31:53 +02:00
Sarah Hoffmann
75139961a3 use constant expressions to select partial indexes in reverse
When expressions are generated with SQLAlchemy, any constants are
replaced with bind parameters. The bind parameters become parameters of
prepared statements. The result is that the query planner tends to
oversee that the partial indexes can be used.
2023-07-09 00:31:53 +02:00
Sarah Hoffmann
3e2dd59a94 remove disabling of prepared statements
Fixes for query planning with prepared statements to follow in
subsequent commits.
2023-07-08 15:59:15 +02:00
Sarah Hoffmann
c01386b5b4 Merge pull request #3107 from lonvia/performance-search
Assorted fixes to Python search code
2023-07-08 15:53:42 +02:00
Sarah Hoffmann
9cb8447673 increase minimum required SQLAlchemy version to 1.4.31 2023-07-06 14:15:51 +02:00
Sarah Hoffmann
e67355ab0e make get_addressdata calls cachable
VALUEs() is not a cachable construct in SQLAlchemy, so use arrays
instead. Also add a special case for single results, the usual result
for reverse queries.
2023-07-06 12:16:57 +02:00
Sarah Hoffmann
9cb9b670d1 add support for postcompile literals in debug output 2023-07-06 12:16:57 +02:00
Sarah Hoffmann
3e725bb2db fix SRID handling in Geometry type 2023-07-06 12:16:57 +02:00
Sarah Hoffmann
cc45930ef9 avoid lookup via partials on frequent words
Drops expensive searches via partials on terms like 'rue de'.

See #2979.
2023-07-06 12:16:57 +02:00
Sarah Hoffmann
3266daa8fd add a small penalty to lookups in address vectors 2023-07-04 16:54:42 +02:00
Sarah Hoffmann
ce17b0eeca Merge pull request #3101 from lonvia/custom-geometry-type
Improve use of SQLAlchemy statement cache with search queries
2023-07-03 11:03:26 +02:00
Sarah Hoffmann
17a65d82bb make types compatible with older Python versions 2023-07-02 23:06:42 +02:00
Sarah Hoffmann
cc7646665c remove GeoAlchemy as dependency 2023-07-02 23:06:42 +02:00
Sarah Hoffmann
82216ebf8b always run function update on migrations
This means that we can have migrations which require nothing but
an update of the functions.
2023-07-01 20:18:59 +02:00
Sarah Hoffmann
49e0d83d5d fix linting issues 2023-07-01 20:18:59 +02:00
Sarah Hoffmann
673c3c7a55 replace regexp_match with generic op() functions
Works around a bug in SQLAlchemy where regexp_match creates an
unstable cache key.
2023-07-01 18:15:22 +02:00
Sarah Hoffmann
5135041405 replace CASE construct with plpgsql function 2023-07-01 18:15:22 +02:00
Sarah Hoffmann
42631b85c7 band-aid for SQLAlchemy 1.4 2023-07-01 18:03:07 +02:00
Sarah Hoffmann
9f6f12cfeb move search to bind parameters 2023-07-01 18:03:07 +02:00
Sarah Hoffmann
6c4c9ec1f2 switch reverse() to new Geometry datatype
Also switches to using bind parameters for recurring parameters.
2023-07-01 18:03:07 +02:00
Sarah Hoffmann
4bb4db0668 introduce slim Geometry database type 2023-07-01 18:03:07 +02:00
Sarah Hoffmann
505fdd02ca Merge pull request #3100 from lonvia/fix-name-merging-in-unnamed-boundaries
Fix merging of linked names into unnamed boundaries
2023-07-01 15:36:56 +02:00
Sarah Hoffmann
a873f260cf fix merging of linked names into unnamed boundaries
The NULL value of the boundaries' name field was erasing all
content when used in SQL operations.
2023-06-30 22:14:11 +02:00
Sarah Hoffmann
b45f761227 Merge pull request #3099 from lonvia/determine-place-address-from-tokenizer
Use information from tokenizer to determine street vs. place address
2023-06-30 21:47:57 +02:00
Sarah Hoffmann
d7a3039c2a also switch legacy tokenizer to new street/place choice behaviour 2023-06-30 17:03:17 +02:00
Sarah Hoffmann
6c5589c9d2 fix optional string representation or repr(PlaceName) 2023-06-30 11:10:44 +02:00
Sarah Hoffmann
645ea5a057 use information from tokenizer to determine street vs. place address
So far the SQL logic used the information from the address field
to determine if an address is attached to a street or place.
This changes the logic to use the information provided in the
token_info. This allows sanitizers to enforce a certain parenting
without changing the visible address information.
2023-06-30 11:08:25 +02:00
Sarah Hoffmann
2755ebe883 Merge pull request #3094 from lonvia/fix-failing-bdd-tests
Add BDD tests against Python frontend to CI
2023-06-22 22:28:31 +02:00
Sarah Hoffmann
4b829b5ff9 Merge pull request #3090 from mtmail/check-database-on-frozen-database
check-database on frozen db shouldnt recommend indexing
2023-06-22 20:11:30 +02:00
Sarah Hoffmann
ed19340af0 add python frontend tests to CI 2023-06-22 17:29:44 +02:00
Sarah Hoffmann
2d05ff0190 slightly adapt postcode tests 2023-06-22 16:51:59 +02:00
Sarah Hoffmann
0d338fa4c0 bdd: fix faking HTTP headers for python web frameworks 2023-06-22 14:00:33 +02:00
mtmail
15a66e7b7d Merge branch 'osm-search:master' into check-database-on-frozen-database 2023-06-22 12:14:55 +02:00
Sarah Hoffmann
3a21999a17 move text normalization into extra function 2023-06-22 10:48:05 +02:00
Sarah Hoffmann
08dcd05d7b Merge pull request #3093 from lonvia/remove-sanic
Remove support for Sanic
2023-06-22 09:55:32 +02:00