Darkshredder
b108bd1c1e
Linting fix
2021-03-12 18:28:47 +05:30
Darkshredder
077a8c1f95
refactored tests and made changes to code for easy readibility
2021-03-12 18:23:20 +05:30
Darkshredder
7a874d5b97
Ported createCountryNames() to python and added tests
2021-03-12 10:28:41 +05:30
Sarah Hoffmann
9086a794a1
Merge pull request #2204 from darkshredder/tiger-data
...
Ported tiger-data-import to Python and Added Tarball Support
2021-03-11 22:48:38 +01:00
Darkshredder
e5719de657
Added fixture for sql_preprocessor and fixed some issues
2021-03-11 15:39:17 +05:30
Darkshredder
8486a83cf5
Added test for tarfile
2021-03-10 18:14:17 +05:30
Darkshredder
ccfad57fca
Added test and removed runlegacyscript
2021-03-10 17:18:12 +05:30
Darkshredder
64128b699a
fixed linting, refactored threaded sql handling and removed importTigerData() function
2021-03-10 13:28:29 +05:30
Darkshredder
4080fbb95c
Test fixes
2021-03-09 01:00:56 +05:30
Darkshredder
14ec83c886
Linting fixes
2021-03-08 23:10:49 +05:30
Darkshredder
122c4618b9
Linting fixes
2021-03-08 22:59:51 +05:30
Darkshredder
2af82975cd
Ported tiger-data-import to python and Added Tarball Support
2021-03-08 21:57:56 +05:30
Sarah Hoffmann
35f4695b67
Merge pull request #2200 from lonvia/migrations-for-current-version
...
Introduce a command for database migration
2021-03-08 10:14:03 +01:00
Sarah Hoffmann
3c9e09545e
documentation for new migration command
2021-03-06 16:38:37 +01:00
Sarah Hoffmann
764a41b973
automatic migration from 3.6 release
...
Adds a 'admin --migrate' command that checks for the current
database version and runs any necessary migrations. Also
has migrations going back to 3.6.
2021-03-06 16:36:57 +01:00
Sarah Hoffmann
9d103503f7
Merge pull request #2197 from lonvia/use-jinja-for-sql-preprocessing
...
Use jinja2 for SQL preprocessing
2021-03-04 16:36:18 +01:00
Sarah Hoffmann
09f4d767e4
port index creation to python
...
Also switches to jinja-based preprocessing, which allows to
simplify the SQL files. Use 'if not exists' where possible
so that the step can be rerun to fix missing indexes.
2021-03-04 11:11:47 +01:00
Sarah Hoffmann
dd301cf5ac
indexer: ANALYSE must be run outside transactions
2021-03-04 11:06:33 +01:00
Sarah Hoffmann
eacabb0e96
move table creation to jinja-based preprocessing
2021-03-03 22:07:51 +01:00
Sarah Hoffmann
6cda021d9b
add new jinja2 requirement
2021-03-03 17:51:08 +01:00
Sarah Hoffmann
d2bd6aa78d
introduce jinja2 for preprocessing SQL
...
Replaces various hand-crafted replacements of varying format with
a single Jinja2 templating mechanism. Allows full access to
configuration if necessary.
2021-03-03 17:51:08 +01:00
Sarah Hoffmann
6b306f30b6
Merge pull request #2194 from grischard/patch-1
...
Fix typo in .github/actions/build-nominatim/action.yml
2021-03-03 11:34:12 +01:00
Guillaume Rischard
c48fd18344
Update action.yml
2021-03-03 11:20:21 +01:00
Sarah Hoffmann
8ea7e04363
Merge pull request #2192 from lonvia/database-versioning
...
Introduce database versioning
2021-03-02 15:57:46 +01:00
Sarah Hoffmann
32c2d2b248
document new status fields
2021-03-01 22:21:37 +01:00
Sarah Hoffmann
111cca8c9a
return database version with status API
2021-03-01 22:17:16 +01:00
Sarah Hoffmann
7ae9c3a9f0
add database_version setting to tests
2021-03-01 21:49:33 +01:00
Sarah Hoffmann
bf4320a7d6
do not depend on cmdline parameter for creating partition tables
...
The partition numbers in use only depend on the entries in search_name.
2021-03-01 21:28:39 +01:00
Sarah Hoffmann
3a0a4b9175
save software version in the database
...
The version represents the software version that was used to
import the data.
2021-03-01 20:35:15 +01:00
Sarah Hoffmann
4faefe156c
report software version of status call
2021-03-01 16:47:19 +01:00
Sarah Hoffmann
86273f5e2a
introduce database patch level for version
...
This will be needed later for automatic migrations.
2021-03-01 16:46:19 +01:00
Sarah Hoffmann
b4f64aa770
make sure that calls to PHP legacy scripts are fatal on error
2021-03-01 16:10:45 +01:00
Sarah Hoffmann
976c5e9121
introduce table for in-database properties
...
Adds a simple table where settings for the database can be
saved. This is useful for state that must not change after
import.
2021-03-01 16:09:17 +01:00
Sarah Hoffmann
db663dd92f
remove unused import
2021-03-01 09:26:08 +01:00
Sarah Hoffmann
90a5d23016
use tmp_path fixture in config tests
2021-03-01 09:24:04 +01:00
Sarah Hoffmann
99e35d256a
fix typo
2021-03-01 09:07:49 +01:00
Sarah Hoffmann
e14e7c6235
Merge pull request #2186 from lonvia/port-import-to-python
...
Move setup procedure to Python
2021-02-27 12:09:23 +01:00
Sarah Hoffmann
b46adbad22
make sure psql always finishes
...
If an execption is raised by other means, we still have to close
the stdin pipe to psql to make sure that it exits and releases its
connection to the database.
2021-02-27 10:24:40 +01:00
Sarah Hoffmann
afabbeb546
older versions of Postgresql need explicit return type
2021-02-27 09:46:42 +01:00
Sarah Hoffmann
d14a3df10f
do not truncate search_name in reverse-only mode
2021-02-27 09:46:42 +01:00
Sarah Hoffmann
9feb84e426
actions: add psutil dependency
2021-02-26 16:50:09 +01:00
Sarah Hoffmann
c7f40e3cee
fix verbose flag for PHP wrapper scripts
...
The flag must come after the command.
2021-02-26 16:49:32 +01:00
Sarah Hoffmann
dd03aeb966
bdd: use python library where possible
...
Replace calls to PHP scripts with direct calls into the
nominatim Python library where possible. This speed up
tests quite a bit.
2021-02-26 16:14:29 +01:00
Sarah Hoffmann
15b5906790
move setup function to python
...
There are still back-calls to PHP for some of the sub-steps.
These needs some larger refactoring to be moved to Python.
2021-02-26 15:02:39 +01:00
Sarah Hoffmann
3ee8d9fa75
properly close connections of indexer after use
2021-02-26 12:10:54 +01:00
Sarah Hoffmann
57db5819ef
prot load-data function to python
2021-02-25 21:32:40 +01:00
Sarah Hoffmann
3c186f8030
add a function for the intial indexing run
...
Also moves postcodes to fully parallel indexing.
2021-02-25 18:42:54 +01:00
Sarah Hoffmann
db5e78c879
remove unused partitionfunction function
2021-02-25 18:42:54 +01:00
Sarah Hoffmann
c7fd0a7af4
port wikipedia importance functions to python
2021-02-25 18:42:54 +01:00
Sarah Hoffmann
32683f73c7
move import-data option to native python
...
This adds a new dependecy to the Python psutil package.
2021-02-25 18:42:54 +01:00