forked from hans/Nominatim
Merge pull request #3859 from lonvia/fix-entrance-addresses
Move entrances to a separate table
This commit is contained in:
@@ -68,10 +68,11 @@ When Nominatim processes an OSM object, it looks for four kinds of tags:
|
|||||||
The _main tags_ classify what kind of place the OSM object represents. One
|
The _main tags_ classify what kind of place the OSM object represents. One
|
||||||
OSM object can have more than one main tag. In such case one database entry
|
OSM object can have more than one main tag. In such case one database entry
|
||||||
is created for each main tag. _Name tags_ represent searchable names of the
|
is created for each main tag. _Name tags_ represent searchable names of the
|
||||||
place. _Address tags_ are used to compute the address hierarchy of the place.
|
place. _Address tags_ are used to compute the address information of the place.
|
||||||
Address tags are used for searching and for creating a display name of the place.
|
Address tags are used for searching and for creating a display name of the place.
|
||||||
_Extra tags_ are any tags that are not directly related to search but
|
_Extra tags_ are any tags that are not directly related to search but
|
||||||
contain interesting additional information.
|
contain interesting additional information. These are just saved in the database
|
||||||
|
and may be returned with the result [on request](../api/Search.md#output-details).
|
||||||
|
|
||||||
!!! danger
|
!!! danger
|
||||||
Some tags in the extratags category are used by Nominatim to better
|
Some tags in the extratags category are used by Nominatim to better
|
||||||
@@ -426,6 +427,56 @@ is added for extratags.
|
|||||||
already delete the tiger tags with `set_prefilters()` because that
|
already delete the tiger tags with `set_prefilters()` because that
|
||||||
would remove tiger:county before the address tags are processed.
|
would remove tiger:county before the address tags are processed.
|
||||||
|
|
||||||
|
## Filling additional tables
|
||||||
|
|
||||||
|
Most of the OSM objects are saved in the main `place` table for further
|
||||||
|
processing. In addition to that, there are some smaller tables that save
|
||||||
|
specialised information. The content of these tables can be customized as
|
||||||
|
well.
|
||||||
|
|
||||||
|
### Entrance table
|
||||||
|
|
||||||
|
The table `place_entrance` saves information about OSM nodes that represent
|
||||||
|
an entrance. This data is later mingled with buildings and other areas and
|
||||||
|
can be returned [on request](../api/Search.md#output-details). The table
|
||||||
|
saves the type of entrance as well as a set of custom extra tags.
|
||||||
|
|
||||||
|
The function `set_entrance_filter()` can be used to customize the table's
|
||||||
|
content.
|
||||||
|
|
||||||
|
When called without any parameter, then filling the entrance table will be
|
||||||
|
disabled. When called with a preset name, the appropriate preset will be
|
||||||
|
applied.
|
||||||
|
|
||||||
|
To create a custom configuration, call the function
|
||||||
|
with a table with the following fields:
|
||||||
|
|
||||||
|
* __main_tags__ is a list of tags that mark an entrance node. The value of the
|
||||||
|
first tag found in the list will be used as the entrance type.
|
||||||
|
* __extra_include__ is an optional list of tags to be added to the extratags
|
||||||
|
for this entrance. When left out, all tags except for the ones defined
|
||||||
|
in 'main_tags' will be included. To disable saving of extra tags, set
|
||||||
|
this to the empty list.
|
||||||
|
* __extra_exclude__ defines an optional list of tags to drop before including
|
||||||
|
the remaining tags as extratags. Note that the tags defined in 'main_tags'
|
||||||
|
will always be excluded, independently of this setting.
|
||||||
|
|
||||||
|
To have even more fine-grained control over the output, you can also hand
|
||||||
|
in a table with a single field `func` containing a callback for processing
|
||||||
|
entrance information. The callback function receives a single parameter,
|
||||||
|
the [osm2pgsql object](https://osm2pgsql.org/doc/manual.html#processing-callbacks).
|
||||||
|
This object itself must not be modified. The callback should return either
|
||||||
|
`nil` when the object is not an entrance. Or it returns a table with a
|
||||||
|
mandatory `entrance` field containing a string with the type of entrance
|
||||||
|
and an optional `extratags` field with a simple key-value table of extra
|
||||||
|
information.
|
||||||
|
|
||||||
|
##### Presets
|
||||||
|
|
||||||
|
| Name | Description |
|
||||||
|
| :----- | :---------- |
|
||||||
|
| default | Standard configuration used with `full` and `extratags` styles. |
|
||||||
|
|
||||||
## Customizing osm2pgsql callbacks
|
## Customizing osm2pgsql callbacks
|
||||||
|
|
||||||
osm2pgsql expects the flex style to implement three callbacks, one process
|
osm2pgsql expects the flex style to implement three callbacks, one process
|
||||||
|
|||||||
@@ -30,6 +30,7 @@ local ADDRESS_TAGS = {}
|
|||||||
local ADDRESS_FILTER = nil
|
local ADDRESS_FILTER = nil
|
||||||
local EXTRATAGS_FILTER
|
local EXTRATAGS_FILTER
|
||||||
local POSTCODE_FALLBACK = true
|
local POSTCODE_FALLBACK = true
|
||||||
|
local ENTRANCE_FUNCTION = nil
|
||||||
|
|
||||||
-- This file can also be directly require'd instead of running it under
|
-- This file can also be directly require'd instead of running it under
|
||||||
-- the themepark framework. In that case the first parameter is usually
|
-- the themepark framework. In that case the first parameter is usually
|
||||||
@@ -40,37 +41,51 @@ if type(themepark) ~= 'table' then
|
|||||||
themepark = nil
|
themepark = nil
|
||||||
end
|
end
|
||||||
|
|
||||||
-- The single place table.
|
-- The place tables carry the raw OSM information.
|
||||||
local place_table_definition = {
|
local table_definitions = {
|
||||||
name = "place",
|
place = {
|
||||||
ids = { type = 'any', id_column = 'osm_id', type_column = 'osm_type' },
|
ids = { type = 'any', id_column = 'osm_id', type_column = 'osm_type' },
|
||||||
columns = {
|
columns = {
|
||||||
{ column = 'class', type = 'text', not_null = true },
|
{ column = 'class', type = 'text', not_null = true },
|
||||||
{ column = 'type', type = 'text', not_null = true },
|
{ column = 'type', type = 'text', not_null = true },
|
||||||
{ column = 'admin_level', type = 'smallint' },
|
{ column = 'admin_level', type = 'smallint' },
|
||||||
{ column = 'name', type = 'hstore' },
|
{ column = 'name', type = 'hstore' },
|
||||||
{ column = 'address', type = 'hstore' },
|
{ column = 'address', type = 'hstore' },
|
||||||
{ column = 'extratags', type = 'hstore' },
|
{ column = 'extratags', type = 'hstore' },
|
||||||
{ column = 'geometry', type = 'geometry', projection = 'WGS84', not_null = true },
|
{ column = 'geometry', type = 'geometry', projection = 'WGS84', not_null = true },
|
||||||
|
},
|
||||||
|
indexes = {}
|
||||||
},
|
},
|
||||||
data_tablespace = os.getenv("NOMINATIM_TABLESPACE_PLACE_DATA"),
|
place_entrance = {
|
||||||
index_tablespace = os.getenv("NOMINATIM_TABLESPACE_PLACE_INDEX"),
|
ids = { type = 'node', id_column = 'osm_id' },
|
||||||
indexes = {}
|
columns = {
|
||||||
|
{ column = 'type', type = 'text', not_null = true },
|
||||||
|
{ column = 'extratags', type = 'hstore' },
|
||||||
|
{ column = 'geometry', type = 'geometry', projection = 'WGS84', not_null = true }
|
||||||
|
},
|
||||||
|
indexes = {}
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
local insert_row
|
local insert_row = {}
|
||||||
local script_path = debug.getinfo(1, "S").source:match("@?(.*/)")
|
local script_path = debug.getinfo(1, "S").source:match("@?(.*/)")
|
||||||
local PRESETS = loadfile(script_path .. 'presets.lua')()
|
local PRESETS = loadfile(script_path .. 'presets.lua')()
|
||||||
|
|
||||||
if themepark then
|
for table_name, table_definition in pairs(table_definitions) do
|
||||||
themepark:add_table(place_table_definition)
|
table_definition.name = table_name
|
||||||
insert_row = function(columns)
|
table_definition.data_tablespace = os.getenv("NOMINATIM_TABLESPACE_PLACE_DATA")
|
||||||
themepark:insert('place', columns, {}, {})
|
table_definition.index_tablespace = os.getenv("NOMINATIM_TABLESPACE_PLACE_INDEX")
|
||||||
end
|
|
||||||
else
|
if themepark then
|
||||||
local place_table = osm2pgsql.define_table(place_table_definition)
|
themepark:add_table(table_definition)
|
||||||
insert_row = function(columns)
|
insert_row[table_name] = function(columns)
|
||||||
place_table:insert(columns)
|
themepark:insert(table_name, columns, {}, {})
|
||||||
|
end
|
||||||
|
else
|
||||||
|
local place_table = osm2pgsql.define_table(table_definition)
|
||||||
|
insert_row[table_name] = function(columns)
|
||||||
|
place_table:insert(columns)
|
||||||
|
end
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
@@ -434,7 +449,7 @@ function Place:write_row(k, v)
|
|||||||
extratags = nil
|
extratags = nil
|
||||||
end
|
end
|
||||||
|
|
||||||
insert_row{
|
insert_row.place{
|
||||||
class = k,
|
class = k,
|
||||||
type = v,
|
type = v,
|
||||||
admin_level = self.admin_level,
|
admin_level = self.admin_level,
|
||||||
@@ -593,6 +608,16 @@ end
|
|||||||
|
|
||||||
-- Process functions for all data types
|
-- Process functions for all data types
|
||||||
function module.process_node(object)
|
function module.process_node(object)
|
||||||
|
if ENTRANCE_FUNCTION ~= nil then
|
||||||
|
local entrance_info = ENTRANCE_FUNCTION(object)
|
||||||
|
if entrance_info ~= nil then
|
||||||
|
insert_row.place_entrance{
|
||||||
|
type = entrance_info.entrance,
|
||||||
|
extratags = entrance_info.extratags,
|
||||||
|
geometry = object:as_point()
|
||||||
|
}
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
local function geom_func(o)
|
local function geom_func(o)
|
||||||
return o:as_point()
|
return o:as_point()
|
||||||
@@ -917,6 +942,94 @@ function module.set_relation_types(data)
|
|||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
|
function module.set_entrance_filter(data)
|
||||||
|
if type(data) == 'string' then
|
||||||
|
local preset = data
|
||||||
|
data = PRESETS.ENTRACE_TABLE[data]
|
||||||
|
if data == nil then
|
||||||
|
error('Unknown preset for entrance table: ' .. preset)
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
ENTRANCE_FUNCTION = data and data.func
|
||||||
|
|
||||||
|
if data ~= nil and data.main_tags ~= nil and next(data.main_tags) ~= nil then
|
||||||
|
if data.extra_include ~= nil and next(data.extra_include) == nil then
|
||||||
|
-- shortcut: no extra tags requested
|
||||||
|
ENTRANCE_FUNCTION = function(o)
|
||||||
|
for _, v in ipairs(data.main_tags) do
|
||||||
|
if o.tags[v] ~= nil then
|
||||||
|
return {entrance = o.tags[v]}
|
||||||
|
end
|
||||||
|
end
|
||||||
|
return nil
|
||||||
|
end
|
||||||
|
else
|
||||||
|
if data.extra_include ~= nil then
|
||||||
|
local tags = {}
|
||||||
|
for _, v in pairs(data.extra_include) do
|
||||||
|
tags[v] = true
|
||||||
|
end
|
||||||
|
if data.extra_exclude ~= nil then
|
||||||
|
for _, v in pairs(data.extra_exclude) do
|
||||||
|
tags[v] = nil
|
||||||
|
end
|
||||||
|
end
|
||||||
|
for _, v in pairs(data.main_tags) do
|
||||||
|
tags[v] = nil
|
||||||
|
end
|
||||||
|
|
||||||
|
ENTRANCE_FUNCTION = function(o)
|
||||||
|
for _, v in ipairs(data.main_tags) do
|
||||||
|
if o.tags[v] ~= nil then
|
||||||
|
local entrance = o.tags[v]
|
||||||
|
local extra = {}
|
||||||
|
for k, v in pairs(tags) do
|
||||||
|
extra[k] = o.tags[k]
|
||||||
|
end
|
||||||
|
if next(extra) == nil then
|
||||||
|
extra = nil
|
||||||
|
end
|
||||||
|
return {entrance = entrance, extratags = extra}
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
return nil
|
||||||
|
end
|
||||||
|
else
|
||||||
|
local notags = {}
|
||||||
|
if data.extra_exclude ~= nil then
|
||||||
|
for _, v in pairs(data.extra_exclude) do
|
||||||
|
notags[v] = 1
|
||||||
|
end
|
||||||
|
end
|
||||||
|
for _, v in pairs(data.main_tags) do
|
||||||
|
notags[v] = 1
|
||||||
|
end
|
||||||
|
|
||||||
|
ENTRANCE_FUNCTION = function(o)
|
||||||
|
for _, v in ipairs(data.main_tags) do
|
||||||
|
if o.tags[v] ~= nil then
|
||||||
|
local entrance = o.tags[v]
|
||||||
|
local extra = {}
|
||||||
|
for k, v in pairs(o.tags) do
|
||||||
|
if notags[k] ~= 1 then
|
||||||
|
extra[k] = v
|
||||||
|
end
|
||||||
|
end
|
||||||
|
if next(extra) == nil then
|
||||||
|
extra = nil
|
||||||
|
end
|
||||||
|
return {entrance = entrance, extratags = extra}
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
return nil
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
function module.get_taginfo()
|
function module.get_taginfo()
|
||||||
return {main = MAIN_KEYS, name = NAMES, address = ADDRESS_TAGS}
|
return {main = MAIN_KEYS, name = NAMES, address = ADDRESS_TAGS}
|
||||||
|
|||||||
@@ -172,10 +172,6 @@ module.MAIN_TAGS_POIS = function (group)
|
|||||||
no = group,
|
no = group,
|
||||||
yes = group,
|
yes = group,
|
||||||
fire_hydrant = group},
|
fire_hydrant = group},
|
||||||
entrance = {'always',
|
|
||||||
no = group},
|
|
||||||
["routing:entrance"] = {exclude_when_key_present('entrance'),
|
|
||||||
no = group},
|
|
||||||
healthcare = {'fallback',
|
healthcare = {'fallback',
|
||||||
yes = group,
|
yes = group,
|
||||||
no = group},
|
no = group},
|
||||||
@@ -385,4 +381,11 @@ module.EXTRATAGS = {}
|
|||||||
|
|
||||||
module.EXTRATAGS.required = {'wikipedia', 'wikipedia:*', 'wikidata', 'capital'}
|
module.EXTRATAGS.required = {'wikipedia', 'wikipedia:*', 'wikidata', 'capital'}
|
||||||
|
|
||||||
|
-- Defaults for the entrance table
|
||||||
|
|
||||||
|
module.ENTRACE_TABLE = {}
|
||||||
|
|
||||||
|
module.ENTRACE_TABLE.default = {main_tags = {'entrance', 'routing:entrance'},
|
||||||
|
extra_exclude = module.IGNORE_KEYS.metatags}
|
||||||
|
|
||||||
return module
|
return module
|
||||||
|
|||||||
@@ -30,3 +30,5 @@ else
|
|||||||
flex.ignore_keys('name')
|
flex.ignore_keys('name')
|
||||||
flex.ignore_keys('address')
|
flex.ignore_keys('address')
|
||||||
end
|
end
|
||||||
|
|
||||||
|
flex.set_entrance_filter('default')
|
||||||
|
|||||||
@@ -683,11 +683,6 @@ DECLARE
|
|||||||
BEGIN
|
BEGIN
|
||||||
{% if debug %}RAISE WARNING '% % % %',NEW.osm_type,NEW.osm_id,NEW.class,NEW.type;{% endif %}
|
{% if debug %}RAISE WARNING '% % % %',NEW.osm_type,NEW.osm_id,NEW.class,NEW.type;{% endif %}
|
||||||
|
|
||||||
IF NEW.class IN ('routing:entrance', 'entrance') THEN
|
|
||||||
-- We don't need entrance nodes in the placex table.
|
|
||||||
RETURN NULL;
|
|
||||||
END IF;
|
|
||||||
|
|
||||||
NEW.place_id := nextval('seq_place');
|
NEW.place_id := nextval('seq_place');
|
||||||
NEW.indexed_status := 1; --STATUS_NEW
|
NEW.indexed_status := 1; --STATUS_NEW
|
||||||
|
|
||||||
|
|||||||
@@ -634,10 +634,8 @@ DECLARE
|
|||||||
BEGIN
|
BEGIN
|
||||||
osm_ids := '{}';
|
osm_ids := '{}';
|
||||||
FOR entrance in SELECT osm_id, type, geometry, extratags
|
FOR entrance in SELECT osm_id, type, geometry, extratags
|
||||||
FROM place
|
FROM place_entrance
|
||||||
WHERE osm_type = 'N'
|
WHERE osm_id IN (SELECT unnest(nodes) FROM planet_osm_ways WHERE id=osmid)
|
||||||
AND osm_id IN (SELECT unnest(nodes) FROM planet_osm_ways WHERE id=osmid)
|
|
||||||
AND class IN ('routing:entrance', 'entrance')
|
|
||||||
LOOP
|
LOOP
|
||||||
osm_ids := array_append(osm_ids, entrance.osm_id);
|
osm_ids := array_append(osm_ids, entrance.osm_id);
|
||||||
INSERT INTO placex_entrance (place_id, osm_id, type, location, extratags)
|
INSERT INTO placex_entrance (place_id, osm_id, type, location, extratags)
|
||||||
|
|||||||
@@ -120,26 +120,39 @@ def create_postcode_parent_index(conn: Connection, **_: Any) -> None:
|
|||||||
|
|
||||||
@_migration(5, 1, 99, 0)
|
@_migration(5, 1, 99, 0)
|
||||||
def create_placex_entrance_table(conn: Connection, config: Configuration, **_: Any) -> None:
|
def create_placex_entrance_table(conn: Connection, config: Configuration, **_: Any) -> None:
|
||||||
""" Add the placex_entrance table to store entrance nodes
|
""" Add the placex_entrance table to store linked-up entrance nodes
|
||||||
"""
|
"""
|
||||||
sqlp = SQLPreprocessor(conn, config)
|
if not table_exists(conn, 'placex_entrance'):
|
||||||
sqlp.run_string(conn, """
|
sqlp = SQLPreprocessor(conn, config)
|
||||||
-- Table to store location of entrance nodes
|
sqlp.run_string(conn, """
|
||||||
DROP TABLE IF EXISTS placex_entrance;
|
-- Table to store location of entrance nodes
|
||||||
CREATE TABLE placex_entrance (
|
CREATE TABLE placex_entrance (
|
||||||
place_id BIGINT NOT NULL,
|
place_id BIGINT NOT NULL,
|
||||||
osm_id BIGINT NOT NULL,
|
osm_id BIGINT NOT NULL,
|
||||||
type TEXT NOT NULL,
|
type TEXT NOT NULL,
|
||||||
location GEOMETRY(Point, 4326) NOT NULL,
|
location GEOMETRY(Point, 4326) NOT NULL,
|
||||||
extratags HSTORE
|
extratags HSTORE
|
||||||
);
|
);
|
||||||
CREATE UNIQUE INDEX idx_placex_entrance_place_id_osm_id ON placex_entrance
|
CREATE UNIQUE INDEX idx_placex_entrance_place_id_osm_id ON placex_entrance
|
||||||
USING BTREE (place_id, osm_id) {{db.tablespace.search_index}};
|
USING BTREE (place_id, osm_id) {{db.tablespace.search_index}};
|
||||||
GRANT SELECT ON placex_entrance TO "{{config.DATABASE_WEBUSER}}" ;
|
GRANT SELECT ON placex_entrance TO "{{config.DATABASE_WEBUSER}}" ;
|
||||||
|
""")
|
||||||
|
|
||||||
-- Create an index on the place table for lookups to populate the entrance
|
|
||||||
-- table
|
@_migration(5, 1, 99, 1)
|
||||||
CREATE INDEX IF NOT EXISTS idx_placex_entrance_lookup ON place
|
def create_place_entrance_table(conn: Connection, config: Configuration, **_: Any) -> None:
|
||||||
USING BTREE (osm_id)
|
""" Add the place_entrance table to store incomming entrance nodes
|
||||||
WHERE class IN ('routing:entrance', 'entrance');
|
"""
|
||||||
""")
|
if not table_exists(conn, 'place_entrance'):
|
||||||
|
with conn.cursor() as cur:
|
||||||
|
cur.execute("""
|
||||||
|
-- Table to store location of entrance nodes
|
||||||
|
CREATE TABLE place_entrance (
|
||||||
|
osm_id BIGINT NOT NULL,
|
||||||
|
type TEXT NOT NULL,
|
||||||
|
extratags HSTORE,
|
||||||
|
geometry GEOMETRY(Point, 4326) NOT NULL
|
||||||
|
);
|
||||||
|
CREATE UNIQUE INDEX place_entrance_osm_id_idx ON place_entrance
|
||||||
|
USING BTREE (osm_id);
|
||||||
|
""")
|
||||||
|
|||||||
@@ -55,7 +55,7 @@ def parse_version(version: str) -> NominatimVersion:
|
|||||||
return NominatimVersion(*[int(x) for x in parts[:2] + parts[2].split('-')])
|
return NominatimVersion(*[int(x) for x in parts[:2] + parts[2].split('-')])
|
||||||
|
|
||||||
|
|
||||||
NOMINATIM_VERSION = parse_version('5.1.99-0')
|
NOMINATIM_VERSION = parse_version('5.1.99-1')
|
||||||
|
|
||||||
POSTGRESQL_REQUIRED_VERSION = (12, 0)
|
POSTGRESQL_REQUIRED_VERSION = (12, 0)
|
||||||
POSTGIS_REQUIRED_VERSION = (3, 0)
|
POSTGIS_REQUIRED_VERSION = (3, 0)
|
||||||
|
|||||||
@@ -8,8 +8,10 @@ Feature: Entrance nodes are recorded
|
|||||||
Given the places
|
Given the places
|
||||||
| osm | class | type | geometry | extratags |
|
| osm | class | type | geometry | extratags |
|
||||||
| W1 | building | yes | (1,2,3,4,1) | |
|
| W1 | building | yes | (1,2,3,4,1) | |
|
||||||
| N1 | entrance | main | 1 | 'wheelchair': 'yes' |
|
And the entrances
|
||||||
| N2 | entrance | yes | 3 | |
|
| osm | type | geometry | extratags |
|
||||||
|
| N1 | main | 1 | 'wheelchair': 'yes' |
|
||||||
|
| N2 | yes | 3 | |
|
||||||
And the ways
|
And the ways
|
||||||
| id | nodes |
|
| id | nodes |
|
||||||
| 1 | 1,2,3,4,1 |
|
| 1 | 1,2,3,4,1 |
|
||||||
|
|||||||
@@ -18,7 +18,7 @@ Feature: Querying fo postcode variants
|
|||||||
| 10 | | | | 11 |
|
| 10 | | | | 11 |
|
||||||
And the places
|
And the places
|
||||||
| osm | class | type | name | addr+postcode | geometry |
|
| osm | class | type | name | addr+postcode | geometry |
|
||||||
| W1 | highway | path | De Weide | 3993 DX | 10,11 |
|
| W1 | highway | path | De Weide | <postcode> | 10,11 |
|
||||||
When importing
|
When importing
|
||||||
When geocoding "3993 DX"
|
When geocoding "3993 DX"
|
||||||
Then result 0 contains
|
Then result 0 contains
|
||||||
|
|||||||
@@ -17,10 +17,12 @@ Feature: Entrance nodes are recorded
|
|||||||
| W1 | 1 |
|
| W1 | 1 |
|
||||||
Then placex_entrance contains exactly
|
Then placex_entrance contains exactly
|
||||||
| place_id | osm_id | type | location!wkt | extratags |
|
| place_id | osm_id | type | location!wkt | extratags |
|
||||||
When updating places
|
When updating entrances
|
||||||
| osm | class | type | geometry |
|
| osm | type | geometry |
|
||||||
| N1 | entrance | main | 1 |
|
| N1 | main | 1 |
|
||||||
| W1 | building | yes | (1,2,3,4,1) |
|
And updating places
|
||||||
|
| osm | class | type | geometry |
|
||||||
|
| W1 | building | yes | (1,2,3,4,1) |
|
||||||
Then placex contains exactly
|
Then placex contains exactly
|
||||||
| object | place_id |
|
| object | place_id |
|
||||||
| W1 | 1 |
|
| W1 | 1 |
|
||||||
@@ -46,13 +48,15 @@ Feature: Entrance nodes are recorded
|
|||||||
| W1 | 2 |
|
| W1 | 2 |
|
||||||
Then placex_entrance contains exactly
|
Then placex_entrance contains exactly
|
||||||
| place_id | osm_id | type | location!wkt | extratags |
|
| place_id | osm_id | type | location!wkt | extratags |
|
||||||
When updating places
|
When marking for delete N1
|
||||||
|
And updating entrances
|
||||||
|
| osm | type | geometry |
|
||||||
|
| N1 | main | 1 |
|
||||||
|
And updating places
|
||||||
| osm | class | type | geometry |
|
| osm | class | type | geometry |
|
||||||
| N1 | entrance | main | 1 |
|
|
||||||
| W1 | building | yes | (1,2,3,4,1) |
|
| W1 | building | yes | (1,2,3,4,1) |
|
||||||
Then placex contains exactly
|
Then placex contains exactly
|
||||||
| object | place_id |
|
| object | place_id |
|
||||||
| N1 | 1 |
|
|
||||||
| W1 | 2 |
|
| W1 | 2 |
|
||||||
And placex_entrance contains exactly
|
And placex_entrance contains exactly
|
||||||
| place_id | osm_id | type | location!wkt | extratags |
|
| place_id | osm_id | type | location!wkt | extratags |
|
||||||
@@ -64,8 +68,10 @@ Feature: Entrance nodes are recorded
|
|||||||
| 4 | 3 |
|
| 4 | 3 |
|
||||||
Given the places
|
Given the places
|
||||||
| osm | class | type | geometry |
|
| osm | class | type | geometry |
|
||||||
| N1 | entrance | main | 1 |
|
|
||||||
| W1 | building | yes | (1,2,3,4,1) |
|
| W1 | building | yes | (1,2,3,4,1) |
|
||||||
|
And the entrances
|
||||||
|
| osm | type | geometry |
|
||||||
|
| N1 | main | 1 |
|
||||||
And the ways
|
And the ways
|
||||||
| id | nodes |
|
| id | nodes |
|
||||||
| 1 | 1, 2, 3, 4, 1 |
|
| 1 | 1, 2, 3, 4, 1 |
|
||||||
@@ -79,7 +85,7 @@ Feature: Entrance nodes are recorded
|
|||||||
When marking for delete N1
|
When marking for delete N1
|
||||||
And updating places
|
And updating places
|
||||||
| osm | class | type | geometry |
|
| osm | class | type | geometry |
|
||||||
| W1 | building | yes | (2,3,4,2) |
|
| W1 | building | yes | (1,2,3,4,1) |
|
||||||
Then placex contains exactly
|
Then placex contains exactly
|
||||||
| object | place_id |
|
| object | place_id |
|
||||||
| W1 | 1 |
|
| W1 | 1 |
|
||||||
@@ -92,9 +98,11 @@ Feature: Entrance nodes are recorded
|
|||||||
| 4 | 3 |
|
| 4 | 3 |
|
||||||
Given the places
|
Given the places
|
||||||
| osm | class | type | geometry |
|
| osm | class | type | geometry |
|
||||||
| N1 | entrance | main | 1 |
|
|
||||||
| N3 | entrance | yes | 3 |
|
|
||||||
| W1 | building | yes | (1,2,3,4,1) |
|
| W1 | building | yes | (1,2,3,4,1) |
|
||||||
|
Given the entrances
|
||||||
|
| osm | type | geometry |
|
||||||
|
| N1 | main | 1 |
|
||||||
|
| N3 | yes | 3 |
|
||||||
And the ways
|
And the ways
|
||||||
| id | nodes |
|
| id | nodes |
|
||||||
| 1 | 1, 2, 3, 4, 1 |
|
| 1 | 1, 2, 3, 4, 1 |
|
||||||
@@ -109,7 +117,7 @@ Feature: Entrance nodes are recorded
|
|||||||
When marking for delete N1
|
When marking for delete N1
|
||||||
And updating places
|
And updating places
|
||||||
| osm | class | type | geometry |
|
| osm | class | type | geometry |
|
||||||
| W1 | building | yes | (2,3,4,2) |
|
| W1 | building | yes | (1,2,3,4,1) |
|
||||||
Then placex contains exactly
|
Then placex contains exactly
|
||||||
| object | place_id |
|
| object | place_id |
|
||||||
| W1 | 1 |
|
| W1 | 1 |
|
||||||
|
|||||||
124
test/bdd/features/osm2pgsql/import/entrances.feature
Normal file
124
test/bdd/features/osm2pgsql/import/entrances.feature
Normal file
@@ -0,0 +1,124 @@
|
|||||||
|
Feature: Import of entrance objects by osm2pgsql
|
||||||
|
Testing of correct setup of the entrance table
|
||||||
|
|
||||||
|
Scenario: Import simple entrance
|
||||||
|
When loading osm data
|
||||||
|
"""
|
||||||
|
n1 Tshop=sweets,entrance=yes,access=public x4.5 y-4
|
||||||
|
n2 Trouting:entrance=main x66.1 y0.1
|
||||||
|
n3 Tentrance=main,routing:entrance=foot x1 y2
|
||||||
|
n4 Thighway=bus_stop
|
||||||
|
"""
|
||||||
|
Then place contains exactly
|
||||||
|
| object | class | type |
|
||||||
|
| N1 | shop | sweets |
|
||||||
|
| N4 | highway | bus_stop |
|
||||||
|
And place_entrance contains exactly
|
||||||
|
| osm_id | type | extratags!dict | geometry!wkt |
|
||||||
|
| 1 | yes | 'shop': 'sweets', 'access': 'public' | 4.5 -4 |
|
||||||
|
| 2 | main | - | 66.1 0.1 |
|
||||||
|
| 3 | main | - | 1 2 |
|
||||||
|
|
||||||
|
Scenario: Addresses and entrance information can exist on the same node
|
||||||
|
When loading osm data
|
||||||
|
"""
|
||||||
|
n1 Taddr:housenumber=10,addr:street=North,entrance=main
|
||||||
|
"""
|
||||||
|
Then place contains exactly
|
||||||
|
| object | class | type | address+housenumber |
|
||||||
|
| N1 | place | house | 10 |
|
||||||
|
And place_entrance contains exactly
|
||||||
|
| osm_id | type |
|
||||||
|
| 1 | main |
|
||||||
|
Scenario Outline: Entrance import can be disabled
|
||||||
|
Given the lua style file
|
||||||
|
"""
|
||||||
|
local flex = require('import-full')
|
||||||
|
flex.set_entrance_filter<param>
|
||||||
|
"""
|
||||||
|
When loading osm data
|
||||||
|
"""
|
||||||
|
n1 Tentrance=yes,access=public
|
||||||
|
n2 Trouting:entrance=main
|
||||||
|
"""
|
||||||
|
Then place contains exactly
|
||||||
|
| object |
|
||||||
|
And place_entrance contains exactly
|
||||||
|
| osm_id |
|
||||||
|
|
||||||
|
Examples:
|
||||||
|
| param |
|
||||||
|
| () |
|
||||||
|
| (nil) |
|
||||||
|
| {} |
|
||||||
|
| {include={'access'}} |
|
||||||
|
| {main_tags={}} |
|
||||||
|
|
||||||
|
Scenario: Entrance import can have custom main tags
|
||||||
|
Given the lua style file
|
||||||
|
"""
|
||||||
|
local flex = require('import-full')
|
||||||
|
flex.set_entrance_filter{main_tags = {'door'}}
|
||||||
|
"""
|
||||||
|
When loading osm data
|
||||||
|
"""
|
||||||
|
n1 Tentrance=yes,access=public
|
||||||
|
n2 Tdoor=foot,entrance=yes
|
||||||
|
"""
|
||||||
|
Then place contains exactly
|
||||||
|
| object |
|
||||||
|
And place_entrance contains exactly
|
||||||
|
| osm_id | type | extratags!dict |
|
||||||
|
| 2 | foot | 'entrance': 'yes' |
|
||||||
|
|
||||||
|
Scenario: Entrance import can have custom extra tags included
|
||||||
|
Given the lua style file
|
||||||
|
"""
|
||||||
|
local flex = require('import-full')
|
||||||
|
flex.set_entrance_filter{main_tags = {'entrance'},
|
||||||
|
extra_include = {'access'}}
|
||||||
|
"""
|
||||||
|
When loading osm data
|
||||||
|
"""
|
||||||
|
n1 Tentrance=yes,access=public,shop=newspaper
|
||||||
|
n2 Tentrance=yes,shop=sweets
|
||||||
|
"""
|
||||||
|
Then place_entrance contains exactly
|
||||||
|
| osm_id | type | extratags!dict |
|
||||||
|
| 1 | yes | 'access': 'public' |
|
||||||
|
| 2 | yes | - |
|
||||||
|
|
||||||
|
Scenario: Entrance import can have custom extra tags excluded
|
||||||
|
Given the lua style file
|
||||||
|
"""
|
||||||
|
local flex = require('import-full')
|
||||||
|
flex.set_entrance_filter{main_tags = {'entrance', 'door'},
|
||||||
|
extra_exclude = {'shop'}}
|
||||||
|
"""
|
||||||
|
When loading osm data
|
||||||
|
"""
|
||||||
|
n1 Tentrance=yes,access=public,shop=newspaper
|
||||||
|
n2 Tentrance=yes,door=yes,shop=sweets
|
||||||
|
"""
|
||||||
|
Then place_entrance contains exactly
|
||||||
|
| osm_id | type | extratags!dict |
|
||||||
|
| 1 | yes | 'access': 'public' |
|
||||||
|
| 2 | yes | - |
|
||||||
|
|
||||||
|
Scenario: Entrance import can have a custom function
|
||||||
|
Given the lua style file
|
||||||
|
"""
|
||||||
|
local flex = require('import-full')
|
||||||
|
flex.set_entrance_filter{func = function(object)
|
||||||
|
return {entrance='always', extratags = {ref = '1'}}
|
||||||
|
end}
|
||||||
|
"""
|
||||||
|
When loading osm data
|
||||||
|
"""
|
||||||
|
n1 Tentrance=yes,access=public,shop=newspaper
|
||||||
|
n2 Tshop=sweets
|
||||||
|
"""
|
||||||
|
Then place_entrance contains exactly
|
||||||
|
| osm_id | type | extratags!dict |
|
||||||
|
| 1 | always | 'ref': '1' |
|
||||||
|
| 2 | always | 'ref': '1' |
|
||||||
106
test/bdd/features/osm2pgsql/update/entrances.feature
Normal file
106
test/bdd/features/osm2pgsql/update/entrances.feature
Normal file
@@ -0,0 +1,106 @@
|
|||||||
|
Feature: Update of entrance objects by osm2pgsql
|
||||||
|
Testing of correct update of the entrance table
|
||||||
|
|
||||||
|
Scenario: A new entrance is added
|
||||||
|
When loading osm data
|
||||||
|
"""
|
||||||
|
n1 Tshop=shoes
|
||||||
|
"""
|
||||||
|
Then place_entrance contains exactly
|
||||||
|
| osm_id |
|
||||||
|
When updating osm data
|
||||||
|
"""
|
||||||
|
n2 Tentrance=yes
|
||||||
|
"""
|
||||||
|
Then place_entrance contains exactly
|
||||||
|
| osm_id | type |
|
||||||
|
| 2 | yes |
|
||||||
|
|
||||||
|
Scenario: An existing entrance is deleted
|
||||||
|
When loading osm data
|
||||||
|
"""
|
||||||
|
n1 Tentrance=yes
|
||||||
|
"""
|
||||||
|
Then place_entrance contains exactly
|
||||||
|
| osm_id | type |
|
||||||
|
| 1 | yes |
|
||||||
|
When updating osm data
|
||||||
|
"""
|
||||||
|
n1 dD
|
||||||
|
"""
|
||||||
|
Then place_entrance contains exactly
|
||||||
|
| osm_id |
|
||||||
|
|
||||||
|
Scenario: An existing node becomes an entrance
|
||||||
|
When loading osm data
|
||||||
|
"""
|
||||||
|
n1 Tshop=sweets
|
||||||
|
"""
|
||||||
|
Then place_entrance contains exactly
|
||||||
|
| osm_id | type |
|
||||||
|
And place contains exactly
|
||||||
|
| object | class |
|
||||||
|
| N1 | shop |
|
||||||
|
When updating osm data
|
||||||
|
"""
|
||||||
|
n1 Tshop=sweets,entrance=yes
|
||||||
|
"""
|
||||||
|
Then place_entrance contains exactly
|
||||||
|
| osm_id | type |
|
||||||
|
| 1 | yes |
|
||||||
|
And place contains exactly
|
||||||
|
| object | class |
|
||||||
|
| N1 | shop |
|
||||||
|
|
||||||
|
Scenario: An existing entrance tag is removed
|
||||||
|
When loading osm data
|
||||||
|
"""
|
||||||
|
n1 Tshop=sweets,entrance=yes
|
||||||
|
"""
|
||||||
|
Then place_entrance contains exactly
|
||||||
|
| osm_id | type |
|
||||||
|
| 1 | yes |
|
||||||
|
And place contains exactly
|
||||||
|
| object | class |
|
||||||
|
| N1 | shop |
|
||||||
|
When updating osm data
|
||||||
|
"""
|
||||||
|
n1 Tshop=sweets
|
||||||
|
"""
|
||||||
|
Then place_entrance contains exactly
|
||||||
|
| osm_id | type |
|
||||||
|
And place contains exactly
|
||||||
|
| object | class |
|
||||||
|
| N1 | shop |
|
||||||
|
|
||||||
|
Scenario: Extratags are added to an entrance
|
||||||
|
When loading osm data
|
||||||
|
"""
|
||||||
|
n1 Tentrance=yes
|
||||||
|
"""
|
||||||
|
Then place_entrance contains exactly
|
||||||
|
| osm_id | type | extratags |
|
||||||
|
| 1 | yes | - |
|
||||||
|
When updating osm data
|
||||||
|
"""
|
||||||
|
n1 Tentrance=yes,access=yes
|
||||||
|
"""
|
||||||
|
Then place_entrance contains exactly
|
||||||
|
| osm_id | type | extratags!dict |
|
||||||
|
| 1 | yes | 'access': 'yes' |
|
||||||
|
|
||||||
|
Scenario: Extratags are deleted from an entrance
|
||||||
|
When loading osm data
|
||||||
|
"""
|
||||||
|
n1 Tentrance=yes,access=yes
|
||||||
|
"""
|
||||||
|
Then place_entrance contains exactly
|
||||||
|
| osm_id | type | extratags!dict |
|
||||||
|
| 1 | yes | 'access': 'yes' |
|
||||||
|
When updating osm data
|
||||||
|
"""
|
||||||
|
n1 Tentrance=yes
|
||||||
|
"""
|
||||||
|
Then place_entrance contains exactly
|
||||||
|
| osm_id | type | extratags |
|
||||||
|
| 1 | yes | - |
|
||||||
@@ -82,6 +82,21 @@ def import_places(db_conn, named, datatable, node_grid):
|
|||||||
PlaceColumn(node_grid).add_row(datatable[0], row, named is not None).db_insert(cur)
|
PlaceColumn(node_grid).add_row(datatable[0], row, named is not None).db_insert(cur)
|
||||||
|
|
||||||
|
|
||||||
|
@given(step_parse('the entrances'), target_fixture=None)
|
||||||
|
def import_place_entrances(db_conn, datatable, node_grid):
|
||||||
|
""" Insert todo rows into the place_entrance table.
|
||||||
|
"""
|
||||||
|
with db_conn.cursor() as cur:
|
||||||
|
for row in datatable[1:]:
|
||||||
|
data = PlaceColumn(node_grid).add_row(datatable[0], row, False)
|
||||||
|
assert data.columns['osm_type'] == 'N'
|
||||||
|
|
||||||
|
cur.execute("""INSERT INTO place_entrance (osm_id, type, extratags, geometry)
|
||||||
|
VALUES (%s, %s, %s, {})""".format(data.get_wkt()),
|
||||||
|
(data.columns['osm_id'], data.columns['type'],
|
||||||
|
data.columns.get('extratags')))
|
||||||
|
|
||||||
|
|
||||||
@given('the ways', target_fixture=None)
|
@given('the ways', target_fixture=None)
|
||||||
def import_ways(db_conn, datatable):
|
def import_ways(db_conn, datatable):
|
||||||
""" Import raw ways into the osm2pgsql way middle table.
|
""" Import raw ways into the osm2pgsql way middle table.
|
||||||
@@ -151,6 +166,23 @@ def do_update(db_conn, update_config, node_grid, datatable):
|
|||||||
return _collect_place_ids(db_conn)
|
return _collect_place_ids(db_conn)
|
||||||
|
|
||||||
|
|
||||||
|
@when('updating entrances', target_fixture=None)
|
||||||
|
def update_place_entrances(db_conn, datatable, node_grid):
|
||||||
|
""" Insert todo rows into the place_entrance table.
|
||||||
|
"""
|
||||||
|
with db_conn.cursor() as cur:
|
||||||
|
for row in datatable[1:]:
|
||||||
|
data = PlaceColumn(node_grid).add_row(datatable[0], row, False)
|
||||||
|
assert data.columns['osm_type'] == 'N'
|
||||||
|
|
||||||
|
cur.execute("DELETE FROM place_entrance WHERE osm_id = %s",
|
||||||
|
(data.columns['osm_id'],))
|
||||||
|
cur.execute("""INSERT INTO place_entrance (osm_id, type, extratags, geometry)
|
||||||
|
VALUES (%s, %s, %s, {})""".format(data.get_wkt()),
|
||||||
|
(data.columns['osm_id'], data.columns['type'],
|
||||||
|
data.columns.get('extratags')))
|
||||||
|
|
||||||
|
|
||||||
@when('updating postcodes')
|
@when('updating postcodes')
|
||||||
def do_postcode_update(update_config):
|
def do_postcode_update(update_config):
|
||||||
""" Recompute the postcode centroids.
|
""" Recompute the postcode centroids.
|
||||||
@@ -168,6 +200,9 @@ def do_delete_place(db_conn, update_config, node_grid, otype, oid):
|
|||||||
cur.execute('DELETE FROM place WHERE osm_type = %s and osm_id = %s',
|
cur.execute('DELETE FROM place WHERE osm_type = %s and osm_id = %s',
|
||||||
(otype, oid))
|
(otype, oid))
|
||||||
cur.execute('SELECT flush_deleted_places()')
|
cur.execute('SELECT flush_deleted_places()')
|
||||||
|
if otype == 'N':
|
||||||
|
cur.execute('DELETE FROM place_entrance WHERE osm_id = %s',
|
||||||
|
(oid, ))
|
||||||
db_conn.commit()
|
db_conn.commit()
|
||||||
|
|
||||||
cli.nominatim(['index', '-q'], update_config.environ)
|
cli.nominatim(['index', '-q'], update_config.environ)
|
||||||
|
|||||||
@@ -118,6 +118,17 @@ class PlaceColumn:
|
|||||||
else:
|
else:
|
||||||
self.columns[column] = {key: value}
|
self.columns[column] = {key: value}
|
||||||
|
|
||||||
|
def get_wkt(self):
|
||||||
|
if self.columns['osm_type'] == 'N' and self.geometry is None:
|
||||||
|
pt = self.grid.get(str(self.columns['osm_id'])) if self.grid else None
|
||||||
|
if pt is None:
|
||||||
|
pt = (random.uniform(-180, 180), random.uniform(-90, 90))
|
||||||
|
|
||||||
|
return "ST_SetSRID(ST_Point({}, {}), 4326)".format(*pt)
|
||||||
|
|
||||||
|
assert self.geometry is not None, "Geometry missing"
|
||||||
|
return self.geometry
|
||||||
|
|
||||||
def db_delete(self, cursor):
|
def db_delete(self, cursor):
|
||||||
""" Issue a delete for the given OSM object.
|
""" Issue a delete for the given OSM object.
|
||||||
"""
|
"""
|
||||||
@@ -127,17 +138,8 @@ class PlaceColumn:
|
|||||||
def db_insert(self, cursor):
|
def db_insert(self, cursor):
|
||||||
""" Insert the collected data into the database.
|
""" Insert the collected data into the database.
|
||||||
"""
|
"""
|
||||||
if self.columns['osm_type'] == 'N' and self.geometry is None:
|
|
||||||
pt = self.grid.get(str(self.columns['osm_id'])) if self.grid else None
|
|
||||||
if pt is None:
|
|
||||||
pt = (random.uniform(-180, 180), random.uniform(-90, 90))
|
|
||||||
|
|
||||||
self.geometry = "ST_SetSRID(ST_Point({}, {}), 4326)".format(*pt)
|
|
||||||
else:
|
|
||||||
assert self.geometry is not None, "Geometry missing"
|
|
||||||
|
|
||||||
query = 'INSERT INTO place ({}, geometry) values({}, {})'.format(
|
query = 'INSERT INTO place ({}, geometry) values({}, {})'.format(
|
||||||
','.join(self.columns.keys()),
|
','.join(self.columns.keys()),
|
||||||
','.join(['%s' for x in range(len(self.columns))]),
|
','.join(['%s' for x in range(len(self.columns))]),
|
||||||
self.geometry)
|
self.get_wkt())
|
||||||
cursor.execute(query, list(self.columns.values()))
|
cursor.execute(query, list(self.columns.values()))
|
||||||
|
|||||||
Reference in New Issue
Block a user