Sarah Hoffmann
64c591da7f
fix type issues with calls to pyosmium
2022-11-09 20:46:33 +01:00
Sarah Hoffmann
6ddb39fda3
respect socket timeout also in other replication functions
2022-11-09 09:12:37 +01:00
Sarah Hoffmann
1fdcec985a
fix timeout use for replication timeout
...
The timeout parameter is no longer taken into account since
pyosmium switched to the requests library. This adds the parameter
back.
2022-11-09 09:12:37 +01:00
Sarah Hoffmann
abf349fb0d
simplify use of secondary importance
...
The values in the raster are already normalized between 0 and 2**16,
so a simple conversion to [0, 1] will do.
Check for existance of secondary_importance table statically when
creating the SQL function. For that to work importance tables need
to be created before the functions.
2022-10-01 11:01:49 +02:00
Sarah Hoffmann
3185fad918
load views as a SQL file and rename to 'secondary importance'
...
The only requirement for secondary importance is that a raster table
comes out of it. The generic name leaves open where the data comes
from.
2022-10-01 11:01:49 +02:00
Tareq Al-Ahdal
0ab0f0ea44
Integrated OSM views into importance computation
2022-10-01 11:01:49 +02:00
Tareq Al-Ahdal
ac467c7a2d
Enhanced the implementation of OSM views GeoTIFF import functionality
2022-10-01 11:01:49 +02:00
Tareq Al-Ahdal
c85b74497b
Initial implementation of GeoTIFF import functionality
2022-10-01 11:01:49 +02:00
Sarah Hoffmann
a2ee58d8a1
only run analyze on indexing when work was done
...
This speeds up processing when continuing indexing after it was
interrupted.
2022-09-28 10:22:54 +02:00
Sarah Hoffmann
f4d3ae6f70
consolidate indexes over geometry_sectors
...
The index over geometry_sectors are mainly used for ordering
the places which need indexing. That means they function effectively
as a TODO list. Consolodate them so that they always only contain
the places which are still to do. Also add the appropriate index
for the boundary indexing phase.
2022-09-21 10:38:58 +02:00
Sarah Hoffmann
ed3dd81d04
run final index creation in parallel
2022-09-19 11:55:25 +02:00
Sarah Hoffmann
bef1aebf1c
add function for parallel execution of SQL scripts
2022-09-19 11:52:17 +02:00
Sarah Hoffmann
bc94318d83
mypy: fix new warnings due to external type updates
2022-09-05 17:39:35 +02:00
Tareq Al-Ahdal
465d82a92f
Integrated 'collect_os_info.py' into Nominatim's CLI tool
2022-08-13 06:18:10 +08:00
Tareq Al-Ahdal
49f889bf09
Enhanced and refactored 'collect_os_info.py'
...
Changed the script to functional programming paradigm to remove the big number of local attributes to decrease memory usage when running it. Additional OS info are now included.
2022-08-13 06:13:05 +08:00
Sarah Hoffmann
8d082c13e0
adapt to new type annotations from typeshed
...
Some more functions frrom psycopg are now properly annotated.
No ignoring necessary anymore.
2022-08-09 11:06:54 +02:00
Sarah Hoffmann
4fe797d704
remove mypy ignore for psutil.virtual_memory()
...
Now available in typeshed.
2022-08-08 09:44:45 +02:00
Sarah Hoffmann
7d68aa8f04
prepare release 4.1.0
2022-08-05 14:33:11 +02:00
Sarah Hoffmann
b19c90b9a6
export centroid to tokenizer
...
May come in handy when developping sanitizers for an area smaller
than country size.
2022-07-31 22:10:58 +02:00
Sarah Hoffmann
9864b191b1
fix various typos
2022-07-31 17:10:35 +02:00
Sarah Hoffmann
51b6d16dc6
overhaul the token analysis interface
...
The functional split betweenthe two functions is now that the
first one creates the ID that is used in the word table and
the second one creates the variants. There no longer is a
requirement that the ID is the normalized version. We might
later reintroduce the requirement that a normalized version be available
but it doesn't necessarily need to be through the ID.
The function that creates the ID now gets the full PlaceName. That way
it might take into account attributes that were set by the sanitizers.
Finally rename both functions to something more sane.
2022-07-29 15:14:11 +02:00
Sarah Hoffmann
34d27ed45c
move PlaceName into the generic data module
2022-07-29 11:42:20 +02:00
Sarah Hoffmann
094100bbf6
harmonize spelling
...
Stick with the American spelling of Analyze.
2022-07-29 10:52:01 +02:00
Sarah Hoffmann
c8873d34af
harmonize interface of token analysis module
...
The configure() function now receives a Transliterator object instead
of the ICU rules. This harmonizes the parameters with the create
function.
2022-07-29 10:43:07 +02:00
Sarah Hoffmann
f0d640961a
add documentation for custom token analysis
2022-07-29 09:41:28 +02:00
Sarah Hoffmann
3746befd88
add documentation for sanitizer interface
...
Also switches mkdocstrings to 0.18 with the rather unfortunate
consequence that now mkdocstrings-python-legacy is needed as well.
2022-07-28 22:00:29 +02:00
Sarah Hoffmann
d819036daa
add support for external token analysis modules
2022-07-25 16:27:22 +02:00
Sarah Hoffmann
6d41046b15
add support for external sanitizer modules
2022-07-25 16:10:19 +02:00
Sarah Hoffmann
7b7203c149
add function for loading plugin modules
...
Loads modules for configurable code like tokenizers, sanitizers, etc.
Supports internal modules, external libraries and code from the
project directory.
2022-07-25 16:10:10 +02:00
Kian-Meng Ang
f5e52e748f
docs: fix typos
2022-07-20 22:05:31 +08:00
Sarah Hoffmann
5aad105c73
add explicit cast for fetchone
2022-07-18 10:18:51 +02:00
Sarah Hoffmann
83054af46f
remove typing_extensions requirement
...
The typing_extensions package is only necessary now when running mypy.
It won't be used at runtime anymore.
2022-07-18 09:55:58 +02:00
Sarah Hoffmann
a849f3c9ec
add type annotations for command line functions
2022-07-18 09:55:54 +02:00
Sarah Hoffmann
25d854dc5c
add type annotations for Tiger import function
2022-07-18 09:54:29 +02:00
Sarah Hoffmann
9963261d8d
add type annotations to special phrase importer
2022-07-18 09:54:29 +02:00
Sarah Hoffmann
459ab3bbdc
add type annotations to database check functions
2022-07-18 09:54:29 +02:00
Sarah Hoffmann
a21d4d3ac4
add type annotations for database import functions
2022-07-18 09:54:29 +02:00
Sarah Hoffmann
4da1f0da6f
add type annotations for migrations
2022-07-18 09:54:29 +02:00
Sarah Hoffmann
17bbe2637a
add type annotations to tool functions
2022-07-18 09:54:27 +02:00
Sarah Hoffmann
6c6bbe5747
add type annotations for ICU tokenizer
2022-07-18 09:47:57 +02:00
Sarah Hoffmann
18b16e06ca
add type annotations for legacy tokenizer
2022-07-18 09:47:57 +02:00
Sarah Hoffmann
e37cfc64d2
add type annotations to ICU tokenizer helper modules
2022-07-18 09:47:57 +02:00
Sarah Hoffmann
d35e3c25b6
add type annotations for token analysis
...
No annotations for ICU types yet.
2022-07-18 09:47:57 +02:00
Sarah Hoffmann
62eedbb8f6
add type hints for sanitizers
2022-07-18 09:47:57 +02:00
Sarah Hoffmann
5617bffe2f
add type annotations for indexer
2022-07-18 09:47:57 +02:00
Sarah Hoffmann
8adab2c6ca
add typing information for postcode formatter
2022-07-18 09:47:57 +02:00
Sarah Hoffmann
d0c44431d0
add typing information for place_info and country_info
2022-07-18 09:47:57 +02:00
Sarah Hoffmann
282a61ce51
add typing information for utils submodule
2022-07-18 09:47:57 +02:00
Sarah Hoffmann
7a1d22ff15
type annotations for non-blocking DB connection
2022-07-18 09:47:57 +02:00
Sarah Hoffmann
0dff71a410
add type annotations for SQL preprocessor
2022-07-18 09:47:57 +02:00
