Sarah Hoffmann
510eb53f53
test: move Testingcursor into separate class
...
Also adds more convenience functions: counting with a where
statement and a wrapper to execute_values().
2021-05-19 10:30:36 +02:00
Sarah Hoffmann
b2722650d4
do not hide errors when importing tokenizer
...
Explicitly check for the tokenizer source file to check that
the name is correct. We can't use the import error for that
because it hides other import errors like a missing
library.
Fixes #2327 .
2021-05-18 16:28:21 +02:00
Sarah Hoffmann
4abaf71234
add and extend tests for new postcode handling
2021-05-13 14:15:42 +02:00
Sarah Hoffmann
bef300305e
move default country name creation to tokenizer
...
The new function is also used, when a country us updated. All SQL
function related to country names have been removed.
2021-04-30 11:30:51 +02:00
Sarah Hoffmann
ffc2d82b0e
move postcode normalization into tokenizer
2021-04-30 11:30:51 +02:00
Sarah Hoffmann
e1c5673ac3
require tokeinzer for indexer
2021-04-30 11:30:51 +02:00
Sarah Hoffmann
fbbdd31399
move word table and normalisation SQL into tokenizer
...
Creating and populating the word table is now the responsibility
of the tokenizer.
The get_maxwordfreq() function has been replaced with a
simple template parameter to the SQL during function installation.
The number is taken from the parameter list in the database to
ensure that it is not changed after installation.
2021-04-30 11:30:51 +02:00
Sarah Hoffmann
af968d4903
introduce tokenizer modules
...
This adds the boilerplate for selecting configurable tokenizers.
A tokenizer can be chosen at import time and will then install
itself such that it is fixed for the given database import even
when the software itself is updated.
The legacy tokenizer implements Nominatim's traditional algorithms.
2021-04-30 11:29:57 +02:00
Sarah Hoffmann
4fa6c0ad53
simplify constructor for SQL preprocessor
...
Use sql path from config.
2021-04-19 10:26:25 +02:00
Sarah Hoffmann
8f63f9516b
simplify interface for adding tiger data
...
Also simplifies tests using existing fixtures.
2021-04-19 10:26:25 +02:00
Sarah Hoffmann
0f11e311c4
add test for new postcode import function
2021-04-16 16:11:20 +02:00
AntoJvlt
2c19bd5ea3
Encapsulation of tools/special_phrases.py into SpecialPhrasesImporter class and add new tests.
2021-03-25 21:13:57 +01:00
Darkshredder
e5719de657
Added fixture for sql_preprocessor and fixed some issues
2021-03-11 15:39:17 +05:30
Sarah Hoffmann
15b5906790
move setup function to python
...
There are still back-calls to PHP for some of the sub-steps.
These needs some larger refactoring to be moved to Python.
2021-02-26 15:02:39 +01:00
Sarah Hoffmann
c7fd0a7af4
port wikipedia importance functions to python
2021-02-25 18:42:54 +01:00
Sarah Hoffmann
32683f73c7
move import-data option to native python
...
This adds a new dependecy to the Python psutil package.
2021-02-25 18:42:54 +01:00
Sarah Hoffmann
f6e894a53a
port database setup function to python
...
Hide the former PHP functions in a transition command until
they are removed.
2021-02-25 18:42:54 +01:00
Sarah Hoffmann
e520613362
convert connect() into a context manager
2021-02-25 18:42:54 +01:00
Sarah Hoffmann
101a1f895d
port freeze function to python
2021-02-17 21:43:15 +01:00
Sarah Hoffmann
d81e152804
integrate analyse of indexing into nominatim tool
2021-02-08 22:22:49 +01:00
Sarah Hoffmann
4cb6dc01f3
port replication update function to python
2021-01-30 15:50:34 +01:00
Sarah Hoffmann
d78f0ba804
port replication initialisation to Python
2021-01-26 22:50:54 +01:00
Sarah Hoffmann
94fa7162be
port address level computation to Python
...
Also adds simple tests for correct table creation.
2021-01-26 22:50:54 +01:00
Sarah Hoffmann
063a4cb403
cli indexer tests need a fake database
...
The Indexer constructor opens a connection to the given database.
2021-01-20 21:30:27 +01:00
Sarah Hoffmann
eb3b789855
add initial pytest test for Configuration
2021-01-15 14:42:03 +01:00
