safety-screen/Nominatim
1
0
Fork 0
forked from hans/Nominatim
Code Pull Requests Activity

Merge pull request #2932 from lonvia/rework-flex-framework

Browse Source 
Switch to osm2pgsql flex output
This commit is contained in:
Sarah Hoffmann
2022-12-29 17:23:38 +01:00
committed by GitHub
parent 9395c0dadc 610af95ed1
commit 7704b3fc7b
22 changed files with 1189 additions and 1054 deletions
Download Patch File Download Diff File Expand all files Collapse all files

11
CMakeLists.txt
View File

@@ -271,11 +271,12 @@ endif()
install(FILES settings/env.defaults install(FILES settings/env.defaults
settings/address-levels.json settings/address-levels.json
settings/phrase-settings.json settings/phrase-settings.json
settings/import-admin.style settings/import-admin.lua
settings/import-street.style settings/import-street.lua
settings/import-address.style settings/import-address.lua
settings/import-full.style settings/import-full.lua
settings/import-extratags.style settings/import-extratags.lua
settings/flex-base.lua
settings/icu_tokenizer.yaml settings/icu_tokenizer.yaml
settings/country_settings.yaml settings/country_settings.yaml
DESTINATION ${NOMINATIM_CONFIGDIR}) DESTINATION ${NOMINATIM_CONFIGDIR})

482
docs/customize/Import-Styles.md
View File

@@ -1,149 +1,439 @@
## Configuring the Import ## Configuring the Import
Which OSM objects are added to the database and which of the tags are used In the very first step of a Nominatim import, OSM data is loaded into the
can be configured via the import style configuration file. This database. Nominatim uses [osm2pgsql](https://osm2pgsql.org) for this task.
is a JSON file which contains a list of rules which are matched against every It comes with a [flex style](https://osm2pgsql.org/doc/manual.html#the-flex-output)
tag of every object and then assign the tag its specific role. specifically tailored to filter and convert OSM data into Nominatim's
internal data representation.
The style to use is given by the `NOMINATIM_IMPORT_STYLE` configuration There are a number of default configurations for the flex style which
option. There are a number of default styles, which are explained in detail result in geocoding databases of different detail. The
in the [Import section](../admin/Import.md#filtering-imported-data). These [Import section](../admin/Import.md#filtering-imported-data) explains
standard styles may be referenced by their name. these default configurations in detail.
You can also create your own custom style. Put the style file into your You can also create your own custom style. Put the style file into your
project directory and then set `NOMINATIM_IMPORT_STYLE` to the name of the file. project directory and then set `NOMINATIM_IMPORT_STYLE` to the name of the file.
It is always recommended to start with one of the standard styles and customize It is always recommended to start with one of the standard styles and customize
those. You find the standard styles under the name `import-<stylename>.style` those. You find the standard styles under the name `import-<stylename>.lua`
in the standard Nominatim configuration path (usually `/etc/nominatim` or in the standard Nominatim configuration path (usually `/etc/nominatim` or
`/usr/local/etc/nominatim`). `/usr/local/etc/nominatim`).
The remainder of the page describes the format of the file. The remainder of the page describes how the flex style works and how to
customize it.
### Configuration Rules ### The `flex-base.lua` module
A single rule looks like this: The core of Nominatim's flex import configuration is the `flex-base` module.
It defines the table layout used by Nominatim and provides standard
implementations for the import callbacks that make it easy to customize
how OSM tags are used by Nominatim.
Every custom style should include this module to make sure that the correct
tables are created. Thus start your custom style as follows:
``` lua
local flex = require('flex-base')
```json
{
"keys" : ["key1", "key2", ...],
"values" : {
"value1" : "prop",
"value2" : "prop1,prop2"
}
}
``` ```
A rule first defines a list of keys to apply the rule to. This is always a list The following sections explain how the module can be customized.
of strings. The string may have four forms. An empty string matches against
any key. A string that ends in an asterisk `*` is a prefix match and accordingly
matches against any key that starts with the given string (minus the `*`). A
suffix match can be defined similarly with a string that starts with a `*`. Any
other string constitutes an exact match.
The second part of the rules defines a list of values and the properties that
apply to a successful match. Value strings may be either empty, which
means that they match any value, or describe an exact match. Prefix
or suffix matching of values is not possible.
For a rule to match, it has to find a valid combination of keys and values. The ### Changing the recognized tags
resulting property is that of the matched values.
The rules in a configuration file are processed sequentially and the first If you just want to change which OSM tags are recognized during import,
match for each tag wins. then there are a number of convenience functions to set the tag lists used
during the processing.
A rule where key and value are the empty string is special. This defines the !!! warning
fallback when none of the rules match. The fallback is always used as a last There are no built-in defaults for the tag lists, so all the functions
resort when nothing else matches, no matter where the rule appears in the file. need to be called from your style script to fully process the data.
Defining multiple fallback rules is not allowed. What happens in this case, Make sure you start from one of the default style and only modify
is undefined. the data you are interested in. You can also derive your style from an
existing style by importing the appropriate module, e.g.
`local flex = require('import-street')`.
### Tag Properties Many of the following functions take _key match lists_. These lists can
contain three kinds of strings to match against tag keys:
A string that ends in an asterisk `*` is a prefix match and accordingly matches
against any key that starts with the given string (minus the `*`).
A suffix match can be defined similarly with a string that starts with a `*`.
Any other string is matched exactly against tag keys.
One or more of the following properties may be given for each tag:
* `main` #### `set_main_tags()` - principal tags
A principal tag. A new row will be added for the object with key and value If a principal or main tag is found on an OSM object, then the object
as `class` and `type`. is included in Nominatim's search index. A single object may also have
multiple main tags. In that case, the object will be included multiple
times in the index, once for each main tag.
* `with_name` The flex script distinguishes between four types of main tags:
When the tag is a principal tag (`main` property set): only really add a new * __always__: a main tag that is used unconditionally
row, if there is any name tag found (a reference tag is not sufficient, see * __named__: consider this main tag only, if the object has a proper name
below). (a reference is not enough, see below).
* __named_with_key__: consider this main tag only, when the object has
a proper name with a domain prefix. For example, if the main tag is
`bridge=yes`, then it will only be added as an extra row, if there is
a tag `bridge:name[:XXX]` for the same object. If this property is set,
all other names that are not domain-specific are ignored.
* __fallback__: use this main tag only, if there is no other main tag.
Fallback always implied `named`, i.e. fallbacks are only tried for
named objects.
* `with_name_key` The `set_main_tags()` function takes exactly one table parameter which
defines the keys and key/value combinations to include and the kind of
main tag. Each lua table key defines an OSM tag key. The value may
be a string defining the kind of main key as described above. Then the tag will
be considered a main tag for any possible value. To further restrict
which values are acceptable, give a table with the permitted values
and their kind of main tag. If the table contains a simple value without
key, then this is used as default for values that are not listed.
When the tag is a principal tag (`main` property set): only really add a new !!! example
row, if there is also a name tag that matches the key of the principal tag. ``` lua
For example, if the main tag is `bridge=yes`, then it will only be added as local flex = require('import-full')
an extra row, if there is a tag `bridge:name[:XXX]` for the same object.
If this property is set, all other names that are not domain-specific are
ignored.
* `fallback` flex.set_main_tags{
boundary = {administrative = 'named'},
highway = {'always', street_lamp = 'named'},
landuse = 'fallback'
}
```
When the tag is a principal tag (`main` property set): only really add a new In this example an object with a `boundary` tag will only be included
row, when no other principal tags for this object have been found. Only one when it has a value of `administrative`. Objects with `highway` tags are
fallback tag can win for an object. always included. However when the value is `street_lamp` then the object
must have a name, too. With any other value, the object is included
independently of the name. Finally, if a `landuse` tag is present then
it will be used independely of the concrete value if neither boundary
nor highway tags were found and the object is named.
* `operator`
When the tag is a principal tag (`main` property set): also include the #### `set_prefilters()` - ignoring tags
`operator` tag in the list of names. This is a special construct for an
out-dated tagging practise in OSM. Fuel stations and chain restaurants
in particular used to have the name of the chain tagged as `operator`.
These days the chain can be more commonly found in the `brand` tag but
there is still enough old data around to warrant this special case.
* `name` Pre-filtering of tags allows to ignore them for any further processing.
Thus pre-filtering takes precedence over any other tag processing. This is
useful when some specific key/value combinations need to be excluded from
processing. When tags are filtered, they may either be deleted completely
or moved to `extratags`. Extra tags are saved with the object and returned
to the user when requested, but are not used otherwise.
Add tag to the list of names. `set_prefilters()` takes a table with four optional fields:
* `ref` * __delete_keys__ is a _key match list_ for tags that should be deleted
* __delete_tags__ contains a table of tag keys pointing to a list of tag
values. Tags with matching key/value pairs are deleted.
* __extra_keys__ is a _key match list_ for tags which should be saved into
extratags
* __delete_tags__ contains a table of tag keys pointing to a list of tag
values. Tags with matching key/value pairs are moved to extratags.
Add tag to the list of names as a reference. At the moment this only means Key list may contain three kinds of strings:
that the object is not considered to be named for `with_name`. A string that ends in an asterisk `*` is a prefix match and accordingly matches
against any key that starts with the given string (minus the `*`).
A suffix match can be defined similarly with a string that starts with a `*`.
Any other string is matched exactly against tag keys.
* `address` !!! example
``` lua
local flex = require('import-full')
Add tag to the list of address tags. If the tag starts with `addr:` or flex.set_prefilters{
`is_in:`, then this prefix is cut off before adding it to the list. delete_keys = {'source', 'source:*'},
extra_tags = {amenity = {'yes', 'no'}}
}
flex.set_main_tags{
amenity = 'always'
}
```
* `postcode` In this example any tags `source` and tags that begin with `source:` are
deleted before any other processing is done. Getting rid of frequent tags
this way can speed up the import.
Add the value as a postcode to the address tags. If multiple tags are Tags with `amenity=yes` or `amenity=no` are moved to extratags. Later
candidate for postcodes, one wins out and the others are dropped. all tags with an `amenity` key are made a main tag. This effectively means
that Nominatim will use all amenity tags except for those with value
yes and no.
* `country` #### `set_name_tags()` - defining names
Add the value as a country code to the address tags. The value must be a The flex script distinguishes between two kinds of names:
two letter country code, otherwise it is ignored. If there are multiple
tags that match, then one wins out and the others are dropped.
* `house` * __main__: the primary names make an object fully searchable.
Main tags of type _named_ will only cause the object to be included when
such a primary name is present. Primary names are usually those found
in the `name` tag and its variants.
* __extra__: extra names are still added to the search index but they are
alone not sufficient to make an object named.
If no principle tags can be found for the object, still add the object with `set_name_tags()` takes a table with two optional fields `main` and `extra`.
`class`=`place` and `type`=`house`. Use this for address nodes that have no They take _key match lists_ for main and extra names respectively.
other function.
* `interpolation` !!! example
``` lua
local flex = require('flex-base')
Add this object as an address interpolation (appears as `class`=`place` and flex.set_main_tags{highway = {traffic_light = 'named'}}
`type`=`houses` in the database). flex.set_name_tags{main = {'name', 'name:*'},
extra = {'ref'}
}
```
* `extra` This example creates a search index over traffic lights but will
only include those that have a common name and not those which just
have some reference ID from the city.
Add tag to the list of extra tags. #### `set_address_tags()` - defining address parts
* `skip` Address tags will be used to build up the address of an object.
Skip the tag completely. Useful when a custom default fallback is defined `set_address_tags()` takes a table with arbitrary fields pointing to
or to define exceptions to rules. _key match lists_. To fields have a special meaning:
A rule can define as many of these properties for one match as it likes. For __main__ defines
example, if the property is `"main,extra"` then the tag will open a new row the tags that make a full address object out of the OSM object. This
but also have the tag appear in the list of extra tags. is usually the housenumber or variants thereof. If a main address tag
appears, then the object will always be included, if necessary with a
fallback of `place=house`. If the key has a prefix of `addr:` or `is_in:`
this will be stripped.
__extra__ defines all supplementary tags for addresses, tags like `addr:street`, `addr:city` etc. If the key has a prefix of `addr:` or `is_in:` this will be stripped.
All other fields will be handled as summary fields. If a key matches the
key match list, then its value will be added to the address tags with the
name of the field as key. If multiple tags match, then an arbitrary one
wins.
Country tags are handled slightly special. Only tags with a two-letter code
are accepted, all other values are discarded.
!!! example
``` lua
local flex = require('import-full')
flex.set_address_tags{
main = {'addr:housenumber'},
extra = {'addr:*'},
postcode = {'postal_code', 'postcode', 'addr:postcode'},
country = {'country-code', 'ISO3166-1'}
}
```
In this example all tags which begin with `addr:` will be saved in
the address tag list. If one of the tags is `addr:housenumber`, the
object will fall back to be entered as a `place=house` in the database
unless there is another interested main tag to be found.
Tags with keys `country-code` and `ISO3166-1` are saved with their
value under `country` in the address tag list. The same thing happens
to postcodes, they will always be saved under the key `postcode` thus
normalizing the multitude of keys that are used in the OSM database.
#### `set_unused_handling()` - processing remaining tags
This function defines what to do with tags that remain after all tags
have been classified using the functions above. There are two ways in
which the function can be used:
`set_unused_handling(delete_keys = ..., delete_tags = ...)` deletes all
keys that match the descriptions in the parameters and moves all remaining
tags into the extratags list.
`set_unused_handling(extra_keys = ..., extra_tags = ...)` moves all tags
matching the parameters into the extratags list and then deletes the remaining
tags. For the format of the parameters see the description in `set_prefilters()`
above.
!!! example
``` lua
local flex = require('import-full')
flex.set_address_tags{
main = {'addr:housenumber'},
extra = {'addr:*', 'tiger:county'}
}
flex.set_unused_handling{delete_keys = {'tiger:*'}}
```
In this example all remaining tags except those beginning with `tiger:`
are moved to the extratags list. Note that it is not possible to
already delete the tiger tags with `set_prefilters()` because that
would remove tiger:county before the address tags are processed.
### Customizing osm2pgsql callbacks
osm2pgsql expects the flex style to implement three callbacks, one process
function per OSM type. If you want to implement special handling for
certain OSM types, you can override the default implementations provided
by the flex-base module.
#### Changing the relation types to be handled
The default scripts only allows relations of type `multipolygon`, `boundary`
and `waterway`. To add other types relations, set `RELATION_TYPES` for
the type to the kind of geometry that should be created. The following
kinds of geometries can be used:
* __relation_as_multipolygon__ creates a (Multi)Polygon from the ways in
the relation. If the ways do not form a valid area, then the object is
silently discarded.
* __relation_as_multiline__ creates a (Mutli)LineString from the ways in
the relations. Ways are combined as much as possible without any regards
to their order in the relation.
!!! Example
``` lua
local flex = require('import-full')
flex.RELATION_TYPES['site'] = flex.relation_as_multipolygon
```
With this line relations of `type=site` will be included in the index
according to main tags found. This only works when the site relation
resolves to a valid area. Nodes in the site relation are not part of the
geometry.
#### Adding additional logic to processing functions
The default processing functions are also exported by the flex-base module
as `process_node`, `process_way` and `process_relation`. These can be used
to implement your own processing functions with some additional processing
logic.
!!! Example
``` lua
local flex = require('import-full')
function osm2pgsql.process_relation(object)
if object.tags.boundary ~= 'administrative' or object.tags.admin_level ~= '2' then
flex.process_relation(object)
end
end
```
This example discards all country-level boundaries and uses standard
handling for everything else. This can be useful if you want to use
your own custom country boundaries.
### Customizing the main processing function
The main processing function of the flex style can be found in the function
`process_tags`. This function is called for all OSM object kinds and is
responsible for filtering the tags and writing out the rows into Postgresql.
!!! Example
``` lua
local flex = require('import-full')
local original_process_tags = flex.process_tags
function flex.process_tags(o)
if o.object.tags.highway ~= nil and o.object.tags.access == 'no' then
return
end
original_process_tags(o)
end
```
This example shows the most simple customization of the process_tags function.
It simply adds some additional processing before running the original code.
To do that, first save the original function and then overwrite process_tags
from the module. In this example all highways which are not accessible
by anyone will be ignored.
#### The `Place` class
The `process_tags` function receives a Lua object of `Place` type which comes
with some handy functions to collect the data necessary for geocoding and
writing it into the place table. Always use this object to fill the table.
The Place class has some attributes which you may access read-only:
* __object__ is the original OSM object data handed in by osm2pgsql
* __admin_level__ is the content of the admin_level tag, parsed into an
integer and normalized to a value between 0 and 15
* __has_name__ is a boolean indicating if the object has a full name
* __names__ is a table with the collected list of name tags
* __address__ is a table with the collected list of address tags
* __extratags__ is a table with the collected list of additional tags to save
There are a number of functions to fill these fields. All functions expect
a table parameter with fields as indicated in the description.
Many of these functions expect match functions which are described in detail
further below.
* __delete{match=...}__ removes all tags that match the match function given
in _match_.
* __grab_extratags{match=...}__ moves all tags that match the match function
given in _match_ into extratags. Returns the number of tags moved.
* __clean{delete=..., extra=...}__ deletes all tags that match _delete_ and
moves the ones that match _extra_ into extratags
* __grab_address_parts{groups=...}__ moves matching tags into the address table.
_groups_ must be a group match function. Tags of the group `main` and
`extra` are added to the address table as is but with `addr:` and `is_in:`
prefixes removed from the tag key. All other groups are added with the
group name as key and the value from the tag. Multiple values of the same
group overwrite each other. The function returns the number of tags saved
from the main group.
* __grab_main_parts{groups=...}__ moves matching tags into the name table.
_groups_ must be a group match function. If a tags of the group `main` is
present, the object will be marked as having a name. Tags of group `house`
produce a fallback to `place=house`. This fallback is return by the function
if present.
There are two functions to write a row into the place table. Both functions
expect the main tag (key and value) for the row and then use the collected
information from the name, address, extratags etc. fields to complete the row.
They also have a boolean parameter `save_extra_mains` which defines how any
unprocessed tags are handled: when True, the tags will be saved as extratags,
when False, they will be simply discarded.
* __write_row(key, value, save_extra_mains)__ creates a new table row from
the current state of the Place object.
* __write_place(key, value, mtype, save_extra_mains)__ creates a new row
conditionally. When value is nil, the function will attempt to look up the
value in the object tags. If value is still nil or mtype is nil, the row
is ignored. An mtype of `always` will then always write out the row,
a mtype of `named` only, when the object has a full name. When mtype
is `named_with_key`, the function checks for a domain name, i.e. a name
tag prefixed with the name of the main key. Only if at least one is found,
the row will be written. The names are replaced with the domain names found.
#### Match functions
The Place functions usually expect either a _match function_ or a
_group match function_ to find the tags to apply their function to.
The __match function__ is a Lua function which takes two parameters,
key and value, and returns a boolean to indicate that a tag matches. The
flex-base module has a convenience function `tag_match()` to create such a
function. It takes a table with two optional fields: `keys` takes a key match
list (see above), `tags` takes a table with keys that point to a list of
possible values, thus defining key/value matches.
The __group match function__ is a Lua function which also takes two parameters,
key and value, and returns a string indicating to which group or type they
belong to. The `tag_group()` can be used to create such a function. It expects
a table where the group names are the keys and the values are a key match list.
### Using the gazetteer output of osm2pgsql
Nominatim still allows you to configure the gazetteer output to remain
backwards compatible with older imports. It will be automatically used
when the style file name ends in `.style`. For documentation of the
old import style, please refer to the documentation of older releases
of Nominatim. Do not use the gazetteer output for new imports. There is no
guarantee that new versions of Nominatim are fully compatible with the
gazetteer output.
### Changing the Style of Existing Databases ### Changing the Style of Existing Databases

1
docs/mkdocs.yml
View File

@@ -51,6 +51,7 @@ nav:
markdown_extensions: markdown_extensions:
- codehilite - codehilite
- admonition - admonition
- pymdownx.superfences
- def_list - def_list
- toc: - toc:
permalink: permalink:

2
nominatim/config.py
View File

@@ -172,7 +172,7 @@ class Configuration:
style = getattr(self, 'IMPORT_STYLE') style = getattr(self, 'IMPORT_STYLE')
if style in ('admin', 'street', 'address', 'full', 'extratags'): if style in ('admin', 'street', 'address', 'full', 'extratags'):
return self.config_dir / f'import-{style}.style' return self.config_dir / f'import-{style}.lua'
return self.find_config_file('', 'IMPORT_STYLE') return self.find_config_file('', 'IMPORT_STYLE')

2
nominatim/tools/exec_utils.py
View File

@@ -125,7 +125,7 @@ def run_osm2pgsql(options: Mapping[str, Any]) -> None:
] ]
if str(options['osm2pgsql_style']).endswith('.lua'): if str(options['osm2pgsql_style']).endswith('.lua'):
env['LUA_PATH'] = ';'.join((str(options['osm2pgsql_style_path'] / 'flex-base.lua'), env['LUA_PATH'] = ';'.join((str(options['osm2pgsql_style_path'] / '?.lua'),
os.environ.get('LUAPATH', ';'))) os.environ.get('LUAPATH', ';')))
cmd.extend(('--output', 'flex')) cmd.extend(('--output', 'flex'))
else: else:

2
osm2pgsql

Submodule osm2pgsql updated: 6a5d2500e9...4facd1aea4

325
settings/flex-base.lua
View File

@@ -1,9 +1,19 @@
-- Core functions for Nominatim import flex style. -- Core functions for Nominatim import flex style.
-- --
local module = {}
local PRE_DELETE = nil
local PRE_EXTRAS = nil
local MAIN_KEYS = nil
local NAMES = nil
local ADDRESS_TAGS = nil
local SAVE_EXTRA_MAINS = false
local POSTCODE_FALLBACK = true
-- The single place table. -- The single place table.
place_table = osm2pgsql.define_table{ local place_table = osm2pgsql.define_table{
name = "place", name = "place",
ids = { type = 'any', id_column = 'osm_id', type_column = 'osm_type' }, ids = { type = 'any', id_column = 'osm_id', type_column = 'osm_type' },
columns = { columns = {
@@ -14,7 +24,25 @@ place_table = osm2pgsql.define_table{
{ column = 'address', type = 'hstore' }, { column = 'address', type = 'hstore' },
{ column = 'extratags', type = 'hstore' }, { column = 'extratags', type = 'hstore' },
{ column = 'geometry', type = 'geometry', projection = 'WGS84', not_null = true }, { column = 'geometry', type = 'geometry', projection = 'WGS84', not_null = true },
} },
indexes = {}
}
------------ Geometry functions for relations ---------------------
function module.relation_as_multipolygon(o)
return o:as_multipolygon()
end
function module.relation_as_multiline(o)
return o:as_multilinestring():line_merge()
end
module.RELATION_TYPES = {
multipolygon = module.relation_as_multipolygon,
boundary = module.relation_as_multipolygon,
waterway = module.relation_as_multiline
} }
------------- Place class ------------------------------------------ ------------- Place class ------------------------------------------
@@ -43,6 +71,17 @@ function Place.new(object, geom_func)
return self return self
end end
function Place:clean(data)
for k, v in pairs(self.object.tags) do
if data.delete ~= nil and data.delete(k, v) then
self.object.tags[k] = nil
elseif data.extra ~= nil and data.extra(k, v) then
self.extratags[k] = v
self.object.tags[k] = nil
end
end
end
function Place:delete(data) function Place:delete(data)
if data.match ~= nil then if data.match ~= nil then
for k, v in pairs(self.object.tags) do for k, v in pairs(self.object.tags) do
@@ -69,54 +108,37 @@ function Place:grab_extratags(data)
return count return count
end end
function Place:grab_address(data) local function strip_address_prefix(k)
if k:sub(1, 5) == 'addr:' then
return k:sub(6)
end
if k:sub(1, 6) == 'is_in:' then
return k:sub(7)
end
return k
end
function Place:grab_address_parts(data)
local count = 0 local count = 0
if data.match ~= nil then if data.groups ~= nil then
for k, v in pairs(self.object.tags) do for k, v in pairs(self.object.tags) do
if data.match(k, v) then local atype = data.groups(k, v)
self.object.tags[k] = nil
if data.include_on_name == true then if atype ~= nil then
if atype == 'main' then
self.has_name = true self.has_name = true
end self.address[strip_address_prefix(k)] = v
count = count + 1
if data.out_key ~= nil then elseif atype == 'extra' then
self.address[data.out_key] = v self.address[strip_address_prefix(k)] = v
return 1
end
if k:sub(1, 5) == 'addr:' then
self.address[k:sub(6)] = v
elseif k:sub(1, 6) == 'is_in:' then
self.address[k:sub(7)] = v
else else
self.address[k] = v self.address[atype] = v
end end
count = count + 1
end
end
end
return count
end
function Place:set_address(key, value)
self.address[key] = value
end
function Place:grab_name(data)
local count = 0
if data.match ~= nil then
for k, v in pairs(self.object.tags) do
if data.match(k, v) then
self.object.tags[k] = nil self.object.tags[k] = nil
self.names[k] = v
if data.include_on_name ~= false then
self.has_name = true
end
count = count + 1
end end
end end
end end
@@ -124,13 +146,30 @@ function Place:grab_name(data)
return count return count
end end
function Place:grab_tag(key)
return self.object:grab_tag(key) function Place:grab_name_parts(data)
local fallback = nil
if data.groups ~= nil then
for k, v in pairs(self.object.tags) do
local atype = data.groups(k, v)
if atype ~= nil then
self.names[k] = v
self.object.tags[k] = nil
if atype == 'main' then
self.has_name = true
elseif atype == 'house' then
self.has_name = true
fallback = {'place', 'house', 'always'}
end
end
end
end
return fallback
end end
function Place:tags()
return self.object.tags
end
function Place:write_place(k, v, mtype, save_extra_mains) function Place:write_place(k, v, mtype, save_extra_mains)
if mtype == nil then if mtype == nil then
@@ -214,7 +253,7 @@ function Place:write_row(k, v, save_extra_mains)
end end
function tag_match(data) function module.tag_match(data)
if data == nil or next(data) == nil then if data == nil or next(data) == nil then
return nil return nil
end end
@@ -276,17 +315,72 @@ function tag_match(data)
end end
function module.tag_group(data)
if data == nil or next(data) == nil then
return nil
end
local fullmatches = {}
local key_prefixes = {}
local key_suffixes = {}
for group, tags in pairs(data) do
for _, key in pairs(tags) do
if key:sub(1, 1) == '*' then
if #key > 1 then
if key_suffixes[#key - 1] == nil then
key_suffixes[#key - 1] = {}
end
key_suffixes[#key - 1][key:sub(2)] = group
end
elseif key:sub(#key, #key) == '*' then
if key_prefixes[#key - 1] == nil then
key_prefixes[#key - 1] = {}
end
key_prefixes[#key - 1][key:sub(1, #key - 1)] = group
else
fullmatches[key] = group
end
end
end
return function (k, v)
local val = fullmatches[k]
if val ~= nil then
return val
end
for slen, slist in pairs(key_suffixes) do
if #k >= slen then
val = slist[k:sub(-slen)]
if val ~= nil then
return val
end
end
end
for slen, slist in pairs(key_prefixes) do
if #k >= slen then
val = slist[k:sub(1, slen)]
if val ~= nil then
return val
end
end
end
end
end
-- Process functions for all data types -- Process functions for all data types
function osm2pgsql.process_node(object) function module.process_node(object)
local function geom_func(o) local function geom_func(o)
return o:as_point() return o:as_point()
end end
process_tags(Place.new(object, geom_func)) module.process_tags(Place.new(object, geom_func))
end end
function osm2pgsql.process_way(object) function module.process_way(object)
local function geom_func(o) local function geom_func(o)
local geom = o:as_polygon() local geom = o:as_polygon()
@@ -298,30 +392,24 @@ function osm2pgsql.process_way(object)
return geom return geom
end end
process_tags(Place.new(object, geom_func)) module.process_tags(Place.new(object, geom_func))
end end
function relation_as_multipolygon(o) function module.process_relation(object)
return o:as_multipolygon() local geom_func = module.RELATION_TYPES[object.tags.type]
end
function relation_as_multiline(o)
return o:as_multilinestring():line_merge()
end
function osm2pgsql.process_relation(object)
local geom_func = RELATION_TYPES[object.tags.type]
if geom_func ~= nil then if geom_func ~= nil then
process_tags(Place.new(object, geom_func)) module.process_tags(Place.new(object, geom_func))
end end
end end
function process_tags(o) -- The process functions are used by default by osm2pgsql.
local fallback osm2pgsql.process_node = module.process_node
osm2pgsql.process_way = module.process_way
osm2pgsql.process_relation = module.process_relation
o:delete{match = PRE_DELETE} function module.process_tags(o)
o:grab_extratags{match = PRE_EXTRAS} o:clean{delete = PRE_DELETE, extra = PRE_EXTRAS}
-- Exception for boundary/place double tagging -- Exception for boundary/place double tagging
if o.object.tags.boundary == 'administrative' then if o.object.tags.boundary == 'administrative' then
@@ -330,54 +418,93 @@ function process_tags(o)
end} end}
end end
-- name keys
local fallback = o:grab_name_parts{groups=NAMES}
-- address keys -- address keys
o:grab_address{match=COUNTRY_TAGS, out_key='country'} if o:grab_address_parts{groups=ADDRESS_TAGS} > 0 and fallback == nil then
fallback = {'place', 'house', 'always'}
end
if o.address.country ~= nil and #o.address.country ~= 2 then if o.address.country ~= nil and #o.address.country ~= 2 then
o.address['country'] = nil o.address['country'] = nil
end end
if o:grab_name{match=HOUSENAME_TAGS} > 0 then if POSTCODE_FALLBACK and fallback == nil and o.address.postcode ~= nil then
fallback = {'place', 'house'} fallback = {'place', 'postcode', 'always'}
end
if o:grab_address{match=HOUSENUMBER_TAGS, include_on_name = true} > 0 and fallback == nil then
fallback = {'place', 'house'}
end
if o:grab_address{match=POSTCODES, out_key='postcode'} > 0 and fallback == nil then
fallback = {'place', 'postcode'}
end end
local is_interpolation = o:grab_address{match=INTERPOLATION_TAGS} > 0 if o.address.interpolation ~= nil then
o:grab_address{match=ADDRESS_TAGS}
if is_interpolation then
o:write_place('place', 'houses', 'always', SAVE_EXTRA_MAINS) o:write_place('place', 'houses', 'always', SAVE_EXTRA_MAINS)
return return
end end
-- name keys o:clean{delete = POST_DELETE, extra = POST_EXTRAS}
o:grab_name{match = NAMES}
o:grab_name{match = REFS, include_on_name = false}
o:delete{match = POST_DELETE}
o:grab_extratags{match = POST_EXTRAS}
-- collect main keys -- collect main keys
local num_mains = 0 for k, v in pairs(o.object.tags) do
for k, v in pairs(o:tags()) do local ktype = MAIN_KEYS[k]
num_mains = num_mains + o:write_place(k, v, MAIN_KEYS[k], SAVE_EXTRA_MAINS) if ktype == 'fallback' then
if o.has_name then
fallback = {k, v, 'named'}
end
elseif ktype ~= nil then
o:write_place(k, v, MAIN_KEYS[k], SAVE_EXTRA_MAINS)
end
end end
if num_mains == 0 then if fallback ~= nil and o.num_entries == 0 then
for tag, mtype in pairs(MAIN_FALLBACK_KEYS) do o:write_place(fallback[1], fallback[2], fallback[3], SAVE_EXTRA_MAINS)
if o:write_place(tag, nil, mtype, SAVE_EXTRA_MAINS) > 0 then end
return end
end
end
if fallback ~= nil then --------- Convenience functions for simple style configuration -----------------
o:write_place(fallback[1], fallback[2], 'always', SAVE_EXTRA_MAINS)
function module.set_prefilters(data)
PRE_DELETE = module.tag_match{keys = data.delete_keys, tags = data.delete_tags}
PRE_EXTRAS = module.tag_match{keys = data.extra_keys,
tags = data.extra_tags}
end
function module.set_main_tags(data)
MAIN_KEYS = data
end
function module.set_name_tags(data)
NAMES = module.tag_group(data)
end
function module.set_address_tags(data)
if data.postcode_fallback ~= nil then
POSTCODE_FALLBACK = data.postcode_fallback
data.postcode_fallback = nil
end
ADDRESS_TAGS = module.tag_group(data)
end
function module.set_unused_handling(data)
if data.extra_keys == nil and data.extra_tags == nil then
POST_DELETE = module.tag_match{keys = data.delete_keys, tags = data.delete_tags}
POST_EXTRAS = nil
SAVE_EXTRA_MAINS = true
elseif data.delete_keys == nil and data.delete_tags == nil then
POST_DELETE = nil
POST_EXTRAS = module.tag_match{keys = data.extra_keys, tags = data.extra_tags}
SAVE_EXTRA_MAINS = false
else
error("unused handler can have only 'extra_keys' or 'delete_keys' set.")
end
end
function set_relation_types(data)
module.RELATION_TYPES = {}
for k, v in data do
if v == 'multipolygon' then
module.RELATION_TYPES[k] = module.relation_as_multipolygon
elseif v == 'multiline' then
module.RELATION_TYPES[k] = module.relation_as_multiline
end end
end end
end end
return module

69
settings/import-address.lua Normal file
View File

@@ -0,0 +1,69 @@
local flex = require('flex-base')
flex.set_main_tags{
highway = {'always',
street_lamp = 'named',
traffic_signals = 'named',
service = 'named',
cycleway = 'named',
path = 'named',
footway = 'named',
steps = 'named',
bridleway = 'named',
track = 'named',
motorway_link = 'named',
trunk_link = 'named',
primary_link = 'named',
secondary_link = 'named',
tertiary_link = 'named'},
boundary = {administrative = 'named',
postal_code = 'named'},
landuse = 'fallback',
place = 'always'
}
flex.set_prefilters{delete_keys = {'building', 'source',
'source', '*source', 'type',
'is_in:postcode', '*:wikidata',
'*:prefix', '*:suffix', 'name:prefix:*', 'name:suffix:*',
'name:etymology', 'name:signed', 'name:botanical',
'addr:street:name', 'addr:street:type'},
delete_tags = {highway = {'no', 'turning_circle', 'mini_roundabout',
'noexit', 'crossing', 'give_way', 'stop'},
landuse = {'cemetry', 'no'},
boundary = {'place'}},
extra_keys = {'wikipedia', 'wikipedia:*', 'wikidata', 'capital', 'area'}
}
flex.set_name_tags{main = {'name', 'name:*',
'int_name', 'int_name:*',
'nat_name', 'nat_name:*',
'reg_name', 'reg_name:*',
'loc_name', 'loc_name:*',
'old_name', 'old_name:*',
'alt_name', 'alt_name:*', 'alt_name_*',
'official_name', 'official_name:*',
'place_name', 'place_name:*',
'short_name', 'short_name:*', 'brand'},
extra = {'ref', 'int_ref', 'nat_ref', 'reg_ref',
'loc_ref', 'old_ref',
'iata', 'icao', 'pcode', 'pcode:*', 'ISO3166-2'},
house = {'addr:housename'}
}
flex.set_address_tags{main = {'addr:housenumber',
'addr:conscriptionnumber',
'addr:streetnumber'},
extra = {'addr:*', 'is_in:*', 'tiger:county'},
postcode = {'postal_code', 'postcode', 'addr:postcode',
'tiger:zip_left', 'tiger:zip_right'},
country = {'country_code', 'ISO3166-1',
'addr:country_code', 'is_in:country_code',
'addr:country', 'is_in:country'},
interpolation = {'addr:interpolation'}
}
flex.set_unused_handling{extra_keys = {'place'}}
return flex

124
settings/import-address.style
View File

@@ -1,124 +0,0 @@
[
{
"keys" : [ "" ],
"values" : {
"no" : "skip"
}
},
{ "keys" : ["wikipedia", "wikipedia:*", "wikidata", "area"],
"values" : {
"" : "extra"
}
},
{
"keys" : ["*:prefix", "*:suffix", "name:prefix:*", "name:suffix:*",
"name:etymology", "name:signed", "name:botanical", "*:wikidata",
"addr:street:name", "addr:street:type"],
"values" : {
"" : "skip"
}
},
{
"keys" : ["ref", "int_ref", "nat_ref", "reg_ref", "loc_ref", "old_ref",
"iata", "icao", "pcode", "ISO3166-2"],
"values" : {
"" : "ref"
}
},
{
"keys" : ["name", "name:*", "int_name", "int_name:*", "nat_name", "nat_name:*",
"reg_name", "reg_name:*", "loc_name", "loc_name:*",
"old_name", "old_name:*", "alt_name", "alt_name:*", "alt_name_*",
"official_name", "official_name:*", "place_name", "place_name:*",
"short_name", "short_name:*", "brand"],
"values" : {
"" : "name"
}
},
{
"keys" : ["landuse"],
"values" : {
"cemetry" : "skip",
"" : "fallback,with_name"
}
},
{
"keys" : ["boundary"],
"values" : {
"administrative" : "main",
"postal_code" : "main"
}
},
{
"keys" : ["place"],
"values" : {
"" : "main"
}
},
{
"keys" : ["addr:housename"],
"values" : {
"" : "name,house"
}
},
{
"keys" : ["addr:housenumber", "addr:conscriptionnumber", "addr:streetnumber"],
"values" : {
"" : "address,house"
}
},
{
"keys" : ["addr:interpolation"],
"values" : {
"" : "interpolation,address"
}
},
{
"keys" : ["postal_code", "postcode", "addr:postcode",
"tiger:zip_left", "tiger:zip_right"],
"values" : {
"" : "postcode,fallback"
}
},
{
"keys" : ["country_code", "ISO3166-1", "is_in:country_code", "is_in:country",
"addr:country", "addr:country_code"],
"values" : {
"" : "country"
}
},
{
"keys" : ["addr:*", "is_in:*", "tiger:county"],
"values" : {
"" : "address"
}
},
{
"keys" : ["highway"],
"values" : {
"motorway" : "main",
"trunk" : "main",
"primary" : "main",
"secondary" : "main",
"tertiary" : "main",
"unclassified" : "main",
"residential" : "main",
"living_street" : "main",
"pedestrian" : "main",
"road" : "main",
"service" : "main,with_name",
"cycleway" : "main,with_name",
"path" : "main,with_name",
"footway" : "main,with_name",
"steps" : "main,with_name",
"bridleway" : "main,with_name",
"track" : "main,with_name",
"byway": "main,with_name",
"motorway_link" : "main,with_name",
"trunk_link" : "main,with_name",
"primary_link" : "main,with_name",
"secondary_link" : "main,with_name",
"tertiary_link" : "main,with_name"
}
}
]

46
settings/import-admin.lua Normal file
View File

@@ -0,0 +1,46 @@
local flex = require('flex-base')
flex.set_main_tags{
boundary = {administrative = 'named'},
landuse = 'fallback',
place = 'always'
}
flex.set_prefilters{delete_keys = {'building', 'source', 'highway',
'addr:housenumber', 'addr:street', 'addr:city',
'source', '*source', 'type',
'is_in:postcode', '*:wikidata',
'*:prefix', '*:suffix', 'name:prefix:*', 'name:suffix:*',
'name:etymology', 'name:signed', 'name:botanical',
'addr:street:name', 'addr:street:type'},
delete_tags = {landuse = {'cemetry', 'no'},
boundary = {'place'}},
extra_keys = {'wikipedia', 'wikipedia:*', 'wikidata', 'capital'}
}
flex.set_name_tags{main = {'name', 'name:*',
'int_name', 'int_name:*',
'nat_name', 'nat_name:*',
'reg_name', 'reg_name:*',
'loc_name', 'loc_name:*',
'old_name', 'old_name:*',
'alt_name', 'alt_name:*', 'alt_name_*',
'official_name', 'official_name:*',
'place_name', 'place_name:*',
'short_name', 'short_name:*', 'brand'},
extra = {'ref', 'int_ref', 'nat_ref', 'reg_ref',
'loc_ref', 'old_ref',
'iata', 'icao', 'pcode', 'pcode:*', 'ISO3166-2'}
}
flex.set_address_tags{extra = {'addr:*', 'is_in:*'},
postcode = {'postal_code', 'postcode', 'addr:postcode'},
country = {'country_code', 'ISO3166-1',
'addr:country_code', 'is_in:country_code',
'addr:country', 'is_in:country'},
postcode_fallback = false
}
flex.set_unused_handling{extra_keys = {'place'}}
return flex

77
settings/import-admin.style
View File

@@ -1,77 +0,0 @@
[
{ "keys" : ["wikipedia", "wikipedia:*", "wikidata"],
"values" : {
"" : "extra"
}
},
{
"keys" : ["*:prefix", "*:suffix", "name:prefix:*", "name:suffix:*",
"name:etymology", "name:signed", "name:botanical", "*:wikidata",
"addr:street:name", "addr:street:type"],
"values" : {
"" : "skip"
}
},
{
"keys" : ["ref", "int_ref", "nat_ref", "reg_ref", "loc_ref", "old_ref",
"iata", "icao", "pcode", "ISO3166-2"],
"values" : {
"" : "ref"
}
},
{
"keys" : ["name", "name:*", "int_name", "int_name:*", "nat_name", "nat_name:*",
"reg_name", "reg_name:*", "loc_name", "loc_name:*",
"old_name", "old_name:*", "alt_name", "alt_name:*", "alt_name_*",
"official_name", "official_name:*", "place_name", "place_name:*",
"short_name", "short_name:*", "brand"],
"values" : {
"" : "name"
}
},
{
"keys" : ["landuse"],
"values" : {
"cemetry" : "skip",
"" : "fallback,with_name"
}
},
{
"keys" : ["boundary"],
"values" : {
"administrative" : "main"
}
},
{
"keys" : ["place"],
"values" : {
"" : "main"
}
},
{
"keys" : ["country_code", "ISO3166-1", "is_in:country_code", "is_in:country",
"addr:country", "addr:country_code"],
"values" : {
"" : "country"
}
},
{
"keys" : ["addr:*", "is_in:*", "tiger:county"],
"values" : {
"" : "address"
}
},
{
"keys" : ["postal_code", "postcode", "addr:postcode",
"tiger:zip_left", "tiger:zip_right"],
"values" : {
"" : "postcode"
}
},
{
"keys" : ["capital"],
"values" : {
"" : "extra"
}
}
]

127
settings/import-extratags.lua
View File

@@ -1,13 +1,9 @@
require('flex-base') local flex = require('flex-base')
RELATION_TYPES = { flex.set_main_tags{
multipolygon = relation_as_multipolygon, building = 'fallback',
boundary = relation_as_multipolygon,
waterway = relation_as_multiline
}
MAIN_KEYS = {
emergency = 'always', emergency = 'always',
healthcare = 'fallback',
historic = 'always', historic = 'always',
military = 'always', military = 'always',
natural = 'named', natural = 'named',
@@ -36,6 +32,8 @@ MAIN_KEYS = {
amenity = 'always', amenity = 'always',
club = 'always', club = 'always',
craft = 'always', craft = 'always',
junction = 'fallback',
landuse = 'fallback',
leisure = 'always', leisure = 'always',
office = 'always', office = 'always',
mountain_pass = 'always', mountain_pass = 'always',
@@ -47,55 +45,42 @@ MAIN_KEYS = {
place = 'always' place = 'always'
} }
MAIN_FALLBACK_KEYS = { flex.set_prefilters{delete_keys = {'note', 'note:*', 'source', '*source', 'attribution',
building = 'named', 'comment', 'fixme', 'FIXME', 'created_by', 'NHD:*',
landuse = 'named', 'nhd:*', 'gnis:*', 'geobase:*', 'KSJ2:*', 'yh:*',
junction = 'named', 'osak:*', 'naptan:*', 'CLC:*', 'import', 'it:fvg:*',
healthcare = 'named' 'type', 'lacounty:*', 'ref:ruian:*', 'building:ruian:type',
} 'ref:linz:*', 'is_in:postcode'},
delete_tags = {emergency = {'yes', 'no', 'fire_hydrant'},
historic = {'yes', 'no'},
PRE_DELETE = tag_match{keys = {'note', 'note:*', 'source', 'source*', 'attribution', military = {'yes', 'no'},
'comment', 'fixme', 'FIXME', 'created_by', 'NHD:*', natural = {'yes', 'no', 'coastline'},
'nhd:*', 'gnis:*', 'geobase:*', 'KSJ2:*', 'yh:*', highway = {'no', 'turning_circle', 'mini_roundabout',
'osak:*', 'naptan:*', 'CLC:*', 'import', 'it:fvg:*', 'noexit', 'crossing', 'give_way', 'stop'},
'type', 'lacounty:*', 'ref:ruian:*', 'building:ruian:type', railway = {'level_crossing', 'no', 'rail'},
'ref:linz:*', 'is_in:postcode'}, man_made = {'survey_point', 'cutline'},
tags = {emergency = {'yes', 'no', 'fire_hydrant'}, aerialway = {'pylon', 'no'},
historic = {'yes', 'no'}, aeroway = {'no'},
military = {'yes', 'no'}, amenity = {'no'},
natural = {'yes', 'no', 'coastline'}, club = {'no'},
highway = {'no', 'turning_circle', 'mini_roundabout', craft = {'no'},
'noexit', 'crossing', 'give_way', 'stop'}, leisure = {'no'},
railway = {'level_crossing', 'no', 'rail'}, office = {'no'},
man_made = {'survey_point', 'cutline'}, mountain_pass = {'no'},
aerialway = {'pylon', 'no'}, shop = {'no'},
aeroway = {'no'}, tourism = {'yes', 'no'},
amenity = {'no'}, bridge = {'no'},
club = {'no'}, tunnel = {'no'},
craft = {'no'}, waterway = {'riverbank'},
leisure = {'no'}, building = {'no'},
office = {'no'}, boundary = {'place'}},
mountain_pass = {'no'}, extra_keys = {'*:prefix', '*:suffix', 'name:prefix:*', 'name:suffix:*',
shop = {'no'},
tourism = {'yes', 'no'},
bridge = {'no'},
tunnel = {'no'},
waterway = {'riverbank'},
building = {'no'},
boundary = {'place'}}
}
POST_DELETE = tag_match{keys = {'tiger:*'}}
PRE_EXTRAS = tag_match{keys = {'*:prefix', '*:suffix', 'name:prefix:*', 'name:suffix:*',
'name:etymology', 'name:signed', 'name:botanical', 'name:etymology', 'name:signed', 'name:botanical',
'wikidata', '*:wikidata', 'wikidata', '*:wikidata',
'addr:street:name', 'addr:street:type'} 'addr:street:name', 'addr:street:type'}
} }
flex.set_name_tags{main = {'name', 'name:*',
NAMES = tag_match{keys = {'name', 'name:*',
'int_name', 'int_name:*', 'int_name', 'int_name:*',
'nat_name', 'nat_name:*', 'nat_name', 'nat_name:*',
'reg_name', 'reg_name:*', 'reg_name', 'reg_name:*',
@@ -104,26 +89,26 @@ NAMES = tag_match{keys = {'name', 'name:*',
'alt_name', 'alt_name:*', 'alt_name_*', 'alt_name', 'alt_name:*', 'alt_name_*',
'official_name', 'official_name:*', 'official_name', 'official_name:*',
'place_name', 'place_name:*', 'place_name', 'place_name:*',
'short_name', 'short_name:*', 'brand'}} 'short_name', 'short_name:*', 'brand'},
extra = {'ref', 'int_ref', 'nat_ref', 'reg_ref',
'loc_ref', 'old_ref',
'iata', 'icao', 'pcode', 'pcode:*', 'ISO3166-2'},
house = {'addr:housename'}
}
REFS = tag_match{keys = {'ref', 'int_ref', 'nat_ref', 'reg_ref', 'loc_ref', 'old_ref', flex.set_address_tags{main = {'addr:housenumber',
'iata', 'icao', 'pcode', 'pcode:*', 'ISO3166-2'}} 'addr:conscriptionnumber',
'addr:streetnumber'},
POSTCODES = tag_match{keys = {'postal_code', 'postcode', 'addr:postcode', extra = {'addr:*', 'is_in:*', 'tiger:county'},
'tiger:zip_left', 'tiger:zip_right'}} postcode = {'postal_code', 'postcode', 'addr:postcode',
'tiger:zip_left', 'tiger:zip_right'},
COUNTRY_TAGS = tag_match{keys = {'country_code', 'ISO3166-1', country = {'country_code', 'ISO3166-1',
'addr:country_code', 'is_in:country_code', 'addr:country_code', 'is_in:country_code',
'addr:country', 'is_in:country'}} 'addr:country', 'is_in:country'},
interpolation = {'addr:interpolation'}
}
HOUSENAME_TAGS = tag_match{keys = {'addr:housename'}}
HOUSENUMBER_TAGS = tag_match{keys = {'addr:housenumber', 'addr:conscriptionnumber', flex.set_unused_handling{delete_keys = {'tiger:*'}}
'addr:streetnumber'}}
INTERPOLATION_TAGS = tag_match{keys = {'addr:interpolation'}}
ADDRESS_TAGS = tag_match{keys = {'addr:*', 'is_in:*', 'tiger:county'}}
SAVE_EXTRA_MAINS = true
return flex

237
settings/import-extratags.style
View File

@@ -1,237 +0,0 @@
[
{
"keys" : ["*source"],
"values" : {
"" : "skip"
}
},
{
"keys" : ["*:prefix", "*:suffix", "name:prefix:*", "name:suffix:*",
"name:etymology", "name:signed", "name:botanical", "wikidata", "*:wikidata",
"addr:street:name", "addr:street:type"],
"values" : {
"" : "extra"
}
},
{
"keys" : ["ref", "int_ref", "nat_ref", "reg_ref", "loc_ref", "old_ref",
"iata", "icao", "pcode", "pcode:*", "ISO3166-2"],
"values" : {
"" : "ref"
}
},
{
"keys" : ["name", "name:*", "int_name", "int_name:*", "nat_name", "nat_name:*",
"reg_name", "reg_name:*", "loc_name", "loc_name:*",
"old_name", "old_name:*", "alt_name", "alt_name:*", "alt_name_*",
"official_name", "official_name:*", "place_name", "place_name:*",
"short_name", "short_name:*", "brand"],
"values" : {
"" : "name"
}
},
{
"keys" : ["addr:housename"],
"values" : {
"" : "name,house"
}
},
{
"keys" : ["emergency"],
"values" : {
"fire_hydrant" : "skip",
"yes" : "skip",
"no" : "skip",
"" : "main"
}
},
{
"keys" : ["historic", "military"],
"values" : {
"no" : "skip",
"yes" : "skip",
"" : "main"
}
},
{
"keys" : ["natural"],
"values" : {
"yes" : "skip",
"no" : "skip",
"coastline" : "skip",
"" : "main,with_name"
}
},
{
"keys" : ["landuse"],
"values" : {
"cemetry" : "main,with_name",
"" : "main,fallback,with_name"
}
},
{
"keys" : ["highway"],
"values" : {
"no" : "skip",
"turning_circle" : "skip",
"mini_roundabout" : "skip",
"noexit" : "skip",
"crossing" : "skip",
"give_way" : "skip",
"stop" : "skip",
"street_lamp" : "main,with_name",
"traffic_signals" : "main,with_name",
"service" : "main,with_name",
"cycleway" : "main,with_name",
"path" : "main,with_name",
"footway" : "main,with_name",
"steps" : "main,with_name",
"bridleway" : "main,with_name",
"track" : "main,with_name",
"byway": "main,with_name",
"motorway_link" : "main,with_name",
"trunk_link" : "main,with_name",
"primary_link" : "main,with_name",
"secondary_link" : "main,with_name",
"tertiary_link" : "main,with_name",
"" : "main"
}
},
{
"keys" : ["railway"],
"values" : {
"level_crossing" : "skip",
"no" : "skip",
"rail" : "extra",
"" : "main,with_name"
}
},
{
"keys" : ["man_made"],
"values" : {
"survey_point" : "skip",
"cutline" : "skip",
"" : "main"
}
},
{
"keys" : ["aerialway"],
"values" : {
"pylon" : "skip",
"no" : "skip",
"" : "main"
}
},
{
"keys" : ["boundary"],
"values" : {
"place" : "skip",
"postal_code" : "main",
"" : "main,with_name"
}
},
{
"keys" : ["aeroway", "amenity", "club", "craft", "leisure",
"office", "mountain_pass"],
"values" : {
"no" : "skip",
"" : "main"
}
},
{
"keys" : ["shop"],
"values" : {
"no" : "skip",
"" : "main"
}
},
{
"keys" : ["tourism"],
"values" : {
"yes" : "skip",
"no" : "skip",
"" : "main"
}
},
{
"keys" : ["bridge", "tunnel"],
"values" : {
"" : "main,with_name_key"
}
},
{
"keys" : ["waterway"],
"values" : {
"riverbank" : "skip",
"" : "main,with_name"
}
},
{
"keys" : ["place"],
"values" : {
"" : "main"
}
},
{
"keys" : ["junction", "healthcare"],
"values" : {
"" : "main,fallback,with_name"
}
},
{
"keys" : ["postal_code", "postcode", "addr:postcode",
"tiger:zip_left", "tiger:zip_right"],
"values" : {
"" : "postcode,fallback"
}
},
{
"keys" : ["country_code", "ISO3166-1", "is_in:country_code", "is_in:country",
"addr:country", "addr:country_code"],
"values" : {
"" : "country"
}
},
{
"keys" : ["addr:housenumber", "addr:conscriptionnumber", "addr:streetnumber"],
"values" : {
"" : "address,house"
}
},
{
"keys" : ["addr:interpolation"],
"values" : {
"" : "interpolation,address"
}
},
{
"keys" : ["addr:*", "is_in:*", "tiger:county"],
"values" : {
"" : "address"
}
},
{
"keys" : ["building"],
"values" : {
"no" : "skip",
"" : "main,fallback,with_name"
}
},
{
"keys" : ["note", "note:*", "source", "source*", "attribution",
"comment", "fixme", "FIXME", "created_by", "tiger:*", "NHD:*",
"nhd:*", "gnis:*", "geobase:*", "KSJ2:*", "yh:*",
"osak:*", "naptan:*", "CLC:*", "import", "it:fvg:*",
"type", "lacounty:*", "ref:ruian:*", "building:ruian:type",
"ref:linz:*"],
"values" : {
"" : "skip"
}
},
{
"keys" : [""],
"values" : {
"" : "extra"
}
}
]

114
settings/import-full.lua Normal file
View File

@@ -0,0 +1,114 @@
local flex = require('flex-base')
flex.set_main_tags{
building = 'fallback',
emergency = 'always',
healthcare = 'fallback',
historic = 'always',
military = 'always',
natural = 'named',
landuse = 'named',
highway = {'always',
street_lamp = 'named',
traffic_signals = 'named',
service = 'named',
cycleway = 'named',
path = 'named',
footway = 'named',
steps = 'named',
bridleway = 'named',
track = 'named',
motorway_link = 'named',
trunk_link = 'named',
primary_link = 'named',
secondary_link = 'named',
tertiary_link = 'named'},
railway = 'named',
man_made = 'always',
aerialway = 'always',
boundary = {'named',
postal_code = 'named'},
aeroway = 'always',
amenity = 'always',
club = 'always',
craft = 'always',
junction = 'fallback',
landuse = 'fallback',
leisure = 'always',
office = 'always',
mountain_pass = 'always',
shop = 'always',
tourism = 'always',
bridge = 'named_with_key',
tunnel = 'named_with_key',
waterway = 'named',
place = 'always'
}
flex.set_prefilters{delete_keys = {'note', 'note:*', 'source', '*source', 'attribution',
'comment', 'fixme', 'FIXME', 'created_by', 'NHD:*',
'nhd:*', 'gnis:*', 'geobase:*', 'KSJ2:*', 'yh:*',
'osak:*', 'naptan:*', 'CLC:*', 'import', 'it:fvg:*',
'type', 'lacounty:*', 'ref:ruian:*', 'building:ruian:type',
'ref:linz:*', 'is_in:postcode'},
delete_tags = {emergency = {'yes', 'no', 'fire_hydrant'},
historic = {'yes', 'no'},
military = {'yes', 'no'},
natural = {'yes', 'no', 'coastline'},
highway = {'no', 'turning_circle', 'mini_roundabout',
'noexit', 'crossing', 'give_way', 'stop'},
railway = {'level_crossing', 'no', 'rail'},
man_made = {'survey_point', 'cutline'},
aerialway = {'pylon', 'no'},
aeroway = {'no'},
amenity = {'no'},
club = {'no'},
craft = {'no'},
leisure = {'no'},
office = {'no'},
mountain_pass = {'no'},
shop = {'no'},
tourism = {'yes', 'no'},
bridge = {'no'},
tunnel = {'no'},
waterway = {'riverbank'},
building = {'no'},
boundary = {'place'}},
extra_keys = {'*:prefix', '*:suffix', 'name:prefix:*', 'name:suffix:*',
'name:etymology', 'name:signed', 'name:botanical',
'wikidata', '*:wikidata',
'addr:street:name', 'addr:street:type'}
}
flex.set_name_tags{main = {'name', 'name:*',
'int_name', 'int_name:*',
'nat_name', 'nat_name:*',
'reg_name', 'reg_name:*',
'loc_name', 'loc_name:*',
'old_name', 'old_name:*',
'alt_name', 'alt_name:*', 'alt_name_*',
'official_name', 'official_name:*',
'place_name', 'place_name:*',
'short_name', 'short_name:*', 'brand'},
extra = {'ref', 'int_ref', 'nat_ref', 'reg_ref',
'loc_ref', 'old_ref',
'iata', 'icao', 'pcode', 'pcode:*', 'ISO3166-2'},
house = {'addr:housename'}
}
flex.set_address_tags{main = {'addr:housenumber',
'addr:conscriptionnumber',
'addr:streetnumber'},
extra = {'addr:*', 'is_in:*', 'tiger:county'},
postcode = {'postal_code', 'postcode', 'addr:postcode',
'tiger:zip_left', 'tiger:zip_right'},
country = {'country_code', 'ISO3166-1',
'addr:country_code', 'is_in:country_code',
'addr:country', 'is_in:country'},
interpolation = {'addr:interpolation'}
}
flex.set_unused_handling{extra_keys = {'place'}}
return flex

240
settings/import-full.style
View File

@@ -1,240 +0,0 @@
[
{
"keys" : ["*source"],
"values" : {
"" : "skip"
}
},
{
"keys" : ["*:prefix", "*:suffix", "name:prefix:*", "name:suffix:*",
"name:etymology", "name:signed", "name:botanical", "wikidata", "*:wikidata",
"addr:street:name", "addr:street:type"],
"values" : {
"" : "extra"
}
},
{
"keys" : ["ref", "int_ref", "nat_ref", "reg_ref", "loc_ref", "old_ref",
"iata", "icao", "pcode", "pcode:*", "ISO3166-2"],
"values" : {
"" : "ref"
}
},
{
"keys" : ["name", "name:*", "int_name", "int_name:*", "nat_name", "nat_name:*",
"reg_name", "reg_name:*", "loc_name", "loc_name:*",
"old_name", "old_name:*", "alt_name", "alt_name:*", "alt_name_*",
"official_name", "official_name:*", "place_name", "place_name:*",
"short_name", "short_name:*", "brand"],
"values" : {
"" : "name"
}
},
{
"keys" : ["addr:housename"],
"values" : {
"" : "name,house"
}
},
{
"keys" : ["emergency"],
"values" : {
"fire_hydrant" : "skip",
"yes" : "skip",
"no" : "skip",
"" : "main"
}
},
{
"keys" : ["historic", "military"],
"values" : {
"no" : "skip",
"yes" : "skip",
"" : "main"
}
},
{
"keys" : ["natural"],
"values" : {
"yes" : "skip",
"no" : "skip",
"coastline" : "skip",
"" : "main,with_name"
}
},
{
"keys" : ["landuse"],
"values" : {
"cemetry" : "main,with_name",
"" : "main,fallback,with_name"
}
},
{
"keys" : ["highway"],
"values" : {
"no" : "skip",
"turning_circle" : "skip",
"mini_roundabout" : "skip",
"noexit" : "skip",
"crossing" : "skip",
"give_way" : "skip",
"stop" : "skip",
"street_lamp" : "main,with_name",
"traffic_signals" : "main,with_name",
"service" : "main,with_name",
"cycleway" : "main,with_name",
"path" : "main,with_name",
"footway" : "main,with_name",
"steps" : "main,with_name",
"bridleway" : "main,with_name",
"track" : "main,with_name",
"byway": "main,with_name",
"motorway_link" : "main,with_name",
"trunk_link" : "main,with_name",
"primary_link" : "main,with_name",
"secondary_link" : "main,with_name",
"tertiary_link" : "main,with_name",
"" : "main"
}
},
{
"keys" : ["railway"],
"values" : {
"level_crossing" : "skip",
"no" : "skip",
"rail" : "skip",
"" : "main,with_name"
}
},
{
"keys" : ["man_made"],
"values" : {
"survey_point" : "skip",
"cutline" : "skip",
"" : "main"
}
},
{
"keys" : ["aerialway"],
"values" : {
"pylon" : "skip",
"no" : "skip",
"" : "main"
}
},
{
"keys" : ["boundary"],
"values" : {
"place" : "skip",
"postal_code" : "main",
"" : "main,with_name"
}
},
{
"keys" : ["aeroway", "amenity", "club", "craft", "leisure",
"office", "mountain_pass"],
"values" : {
"no" : "skip",
"" : "main"
}
},
{
"keys" : ["shop"],
"values" : {
"no" : "skip",
"" : "main"
}
},
{
"keys" : ["tourism"],
"values" : {
"yes" : "skip",
"no" : "skip",
"" : "main"
}
},
{
"keys" : ["bridge", "tunnel"],
"values" : {
"" : "main,with_name_key"
}
},
{
"keys" : ["waterway"],
"values" : {
"riverbank" : "skip",
"" : "main,with_name"
}
},
{
"keys" : ["place"],
"values" : {
"" : "main"
}
},
{
"keys" : ["junction", "healthcare"],
"values" : {
"" : "main,fallback,with_name"
}
},
{
"keys" : ["postal_code", "postcode", "addr:postcode",
"tiger:zip_left", "tiger:zip_right"],
"values" : {
"" : "postcode,fallback"
}
},
{
"keys" : ["country_code", "ISO3166-1", "is_in:country_code", "is_in:country",
"addr:country", "addr:country_code"],
"values" : {
"" : "country"
}
},
{
"keys" : ["addr:housenumber", "addr:conscriptionnumber", "addr:streetnumber"],
"values" : {
"" : "address,house"
}
},
{
"keys" : ["addr:interpolation"],
"values" : {
"" : "interpolation,address"
}
},
{
"keys" : ["addr:*", "is_in:*", "tiger:county"],
"values" : {
"" : "address"
}
},
{
"keys" : ["building"],
"values" : {
"no" : "skip",
"" : "main,fallback,with_name"
}
},
{
"keys" : ["tracktype", "traffic_calming", "service", "cuisine", "capital",
"dispensing", "religion", "denomination", "sport",
"internet_access", "lanes", "surface", "smoothness", "width",
"est_width", "incline", "opening_hours", "collection_times",
"service_times", "disused", "wheelchair", "sac_scale",
"trail_visibility", "mtb:scale", "mtb:description", "wood",
"drive_through", "drive_in", "access", "vehicle", "bicyle",
"foot", "goods", "hgv", "motor_vehicle", "motor_car", "oneway",
"date_on", "date_off", "day_on", "day_off", "hour_on", "hour_off",
"maxweight", "maxheight", "maxspeed", "fee", "toll", "charge",
"population", "description", "image", "attribution", "fax",
"email", "url", "website", "phone", "real_ale", "smoking",
"food", "camera", "brewery", "locality", "wikipedia",
"wikipedia:*", "access:*", "contact:*", "drink:*", "toll:*",
"area"],
"values" : {
"" : "extra"
}
}
]

68
settings/import-street.lua Normal file
View File

@@ -0,0 +1,68 @@
local flex = require('flex-base')
flex.set_main_tags{
highway = {'always',
street_lamp = 'named',
traffic_signals = 'named',
service = 'named',
cycleway = 'named',
path = 'named',
footway = 'named',
steps = 'named',
bridleway = 'named',
track = 'named',
motorway_link = 'named',
trunk_link = 'named',
primary_link = 'named',
secondary_link = 'named',
tertiary_link = 'named'},
boundary = {administrative = 'named'},
landuse = 'fallback',
place = 'always'
}
flex.set_prefilters{delete_keys = {'building', 'source',
'addr:housenumber', 'addr:street',
'source', '*source', 'type',
'is_in:postcode', '*:wikidata',
'*:prefix', '*:suffix', 'name:prefix:*', 'name:suffix:*',
'name:etymology', 'name:signed', 'name:botanical',
'addr:street:name', 'addr:street:type'},
delete_tags = {highway = {'no', 'turning_circle', 'mini_roundabout',
'noexit', 'crossing', 'give_way', 'stop'},
landuse = {'cemetry', 'no'},
boundary = {'place'}},
extra_keys = {'wikipedia', 'wikipedia:*', 'wikidata', 'capital', 'area'}
}
flex.set_name_tags{main = {'name', 'name:*',
'int_name', 'int_name:*',
'nat_name', 'nat_name:*',
'reg_name', 'reg_name:*',
'loc_name', 'loc_name:*',
'old_name', 'old_name:*',
'alt_name', 'alt_name:*', 'alt_name_*',
'official_name', 'official_name:*',
'place_name', 'place_name:*',
'short_name', 'short_name:*', 'brand'},
extra = {'ref', 'int_ref', 'nat_ref', 'reg_ref',
'loc_ref', 'old_ref',
'iata', 'icao', 'pcode', 'pcode:*', 'ISO3166-2'}
}
flex.set_address_tags{main = {'addr:housenumber',
'addr:conscriptionnumber',
'addr:streetnumber'},
extra = {'addr:*', 'is_in:*', 'tiger:county'},
postcode = {'postal_code', 'postcode', 'addr:postcode',
'tiger:zip_left', 'tiger:zip_right'},
country = {'country_code', 'ISO3166-1',
'addr:country_code', 'is_in:country_code',
'addr:country', 'is_in:country'},
interpolation = {'addr:interpolation'},
postcode_fallback = false
}
flex.set_unused_handling{extra_keys = {'place'}}
return flex

92
settings/import-street.style
View File

@@ -1,92 +0,0 @@
[
{ "keys" : ["wikipedia", "wikipedia:*", "wikidata", "area"],
"values" : {
"" : "extra"
}
},
{
"keys" : ["*:prefix", "*:suffix", "name:prefix:*", "name:suffix:*",
"name:etymology", "name:signed", "name:botanical", "*:wikidata",
"addr:street:name", "addr:street:type"],
"values" : {
"" : "skip"
}
},
{
"keys" : ["ref", "int_ref", "nat_ref", "reg_ref", "loc_ref", "old_ref",
"iata", "icao", "pcode", "ISO3166-2"],
"values" : {
"" : "ref"
}
},
{
"keys" : ["name", "name:*", "int_name", "int_name:*", "nat_name", "nat_name:*",
"reg_name", "reg_name:*", "loc_name", "loc_name:*",
"old_name", "old_name:*", "alt_name", "alt_name:*", "alt_name_*",
"official_name", "official_name:*", "place_name", "place_name:*",
"short_name", "short_name:*", "brand"],
"values" : {
"" : "name"
}
},
{
"keys" : ["landuse"],
"values" : {
"cemetry" : "skip",
"" : "fallback,with_name"
}
},
{
"keys" : ["boundary"],
"values" : {
"administrative" : "main"
}
},
{
"keys" : ["place"],
"values" : {
"" : "main"
}
},
{
"keys" : ["country_code", "ISO3166-1", "is_in:country_code", "is_in:country",
"addr:country", "addr:country_code"],
"values" : {
"" : "country"
}
},
{
"keys" : ["addr:*", "is_in:*", "tiger:county"],
"values" : {
"" : "address"
}
},
{
"keys" : ["highway"],
"values" : {
"motorway" : "main",
"trunk" : "main",
"primary" : "main",
"secondary" : "main",
"tertiary" : "main",
"unclassified" : "main",
"residential" : "main",
"living_street" : "main",
"pedestrian" : "main",
"road" : "main",
"service" : "main,with_name",
"cycleway" : "main,with_name",
"path" : "main,with_name",
"footway" : "main,with_name",
"steps" : "main,with_name",
"bridleway" : "main,with_name",
"track" : "main,with_name",
"byway": "main,with_name",
"motorway_link" : "main,with_name",
"trunk_link" : "main,with_name",
"primary_link" : "main,with_name",
"secondary_link" : "main,with_name",
"tertiary_link" : "main,with_name"
}
}
]

200
test/bdd/osm2pgsql/import/custom_style.feature Normal file
View File

@@ -0,0 +1,200 @@
@DB
Feature: Import with custom styles by osm2pgsql
Tests for the example customizations given in the documentation.
Scenario: Custom main tags
Given the lua style file
"""
local flex = require('import-full')
flex.set_main_tags{
boundary = {administrative = 'named'},
highway = {'always', street_lamp = 'named'},
landuse = 'fallback'
}
"""
When loading osm data
"""
n10 Tboundary=administrative x0 y0
n11 Tboundary=administrative,name=Foo x0 y0
n12 Tboundary=electoral x0 y0
n13 Thighway=primary x0 y0
n14 Thighway=street_lamp x0 y0
n15 Thighway=primary,landuse=street x0 y0
"""
Then place contains exactly
| object | class | type |
| N11 | boundary | administrative |
| N13 | highway | primary |
| N15 | highway | primary |
Scenario: Prefiltering tags
Given the lua style file
"""
local flex = require('import-full')
flex.set_prefilters{
delete_keys = {'source', 'source:*'},
extra_tags = {amenity = {'yes', 'no'}}
}
flex.set_main_tags{
amenity = 'always',
tourism = 'always'
}
"""
When loading osm data
"""
n1 Tamenity=yes x0 y6
n2 Tamenity=hospital,source=survey x3 y6
n3 Ttourism=hotel,amenity=yes x0 y0
n4 Ttourism=hotel,amenity=telephone x0 y0
"""
Then place contains exactly
| object | extratags |
| N2:amenity | - |
| N3:tourism | 'amenity': 'yes' |
| N4:tourism | - |
| N4:amenity | - |
Scenario: Name tags
Given the lua style file
"""
local flex = require('flex-base')
flex.set_main_tags{highway = {traffic_light = 'named'}}
flex.set_name_tags{main = {'name', 'name:*'},
extra = {'ref'}
}
"""
When loading osm data
"""
n1 Thighway=stop,name=Something x0 y0
n2 Thighway=traffic_light,ref=453-4 x0 y0
n3 Thighway=traffic_light,name=Greens x0 y0
n4 Thighway=traffic_light,name=Red,ref=45 x0 y0
"""
Then place contains exactly
| object | name |
| N3:highway | 'name': 'Greens' |
| N4:highway | 'name': 'Red', 'ref': '45' |
Scenario: Address tags
Given the lua style file
"""
local flex = require('import-full')
flex.set_address_tags{
main = {'addr:housenumber'},
extra = {'addr:*'},
postcode = {'postal_code', 'postcode', 'addr:postcode'},
country = {'country-code', 'ISO3166-1'}
}
"""
When loading osm data
"""
n1 Ttourism=hotel,addr:street=Foo x0 y0
n2 Taddr:housenumber=23,addr:street=Budd,postal_code=5567 x0 y0
n3 Taddr:street=None,addr:city=Where x0 y0
"""
Then place contains exactly
| object | type | address |
| N1:tourism | hotel | 'street': 'Foo' |
| N2:place | house | 'housenumber': '23', 'street': 'Budd', 'postcode': '5567' |
Scenario: Unused handling
Given the lua style file
"""
local flex = require('import-full')
flex.set_address_tags{
main = {'addr:housenumber'},
extra = {'addr:*', 'tiger:county'}
}
flex.set_unused_handling{delete_keys = {'tiger:*'}}
"""
When loading osm data
"""
n1 Ttourism=hotel,tiger:county=Fargo x0 y0
n2 Ttourism=hotel,tiger:xxd=56,else=other x0 y0
"""
Then place contains exactly
| object | type | address | extratags |
| N1:tourism | hotel | 'tiger:county': 'Fargo' | - |
| N2:tourism | hotel | - | 'else': 'other' |
Scenario: Additional relation types
Given the lua style file
"""
local flex = require('import-full')
flex.RELATION_TYPES['site'] = flex.relation_as_multipolygon
"""
And the grid
| 1 | 2 |
| 4 | 3 |
When loading osm data
"""
n1
n2
n3
n4
w1 Nn1,n2,n3,n4,n1
r1 Ttype=multipolygon,amenity=school Mw1@
r2 Ttype=site,amenity=school Mw1@
"""
Then place contains exactly
| object | type |
| R1:amenity | school |
| R2:amenity | school |
Scenario: Exclude country relations
Given the lua style file
"""
local flex = require('import-full')
function osm2pgsql.process_relation(object)
if object.tags.boundary ~= 'administrative' or object.tags.admin_level ~= '2' then
flex.process_relation(object)
end
end
"""
And the grid
| 1 | 2 |
| 4 | 3 |
When loading osm data
"""
n1
n2
n3
n4
w1 Nn1,n2,n3,n4,n1
r1 Ttype=multipolygon,boundary=administrative,admin_level=4,name=Small Mw1@
r2 Ttype=multipolygon,boundary=administrative,admin_level=2,name=Big Mw1@
"""
Then place contains exactly
| object | type |
| R1:boundary | administrative |
Scenario: Customize processing functions
Given the lua style file
"""
local flex = require('import-full')
local original_process_tags = flex.process_tags
function flex.process_tags(o)
if o.object.tags.highway ~= nil and o.object.tags.access == 'no' then
return
end
original_process_tags(o)
end
"""
When loading osm data
"""
n1 Thighway=residential x0 y0
n2 Thighway=residential,access=no x0 y0
"""
Then place contains exactly
| object | type |
| N1:highway | residential |

8
test/bdd/osm2pgsql/import/tags.feature
View File

@@ -122,10 +122,10 @@ Feature: Tag evaluation
n8003 Tshop=shoes,name:source=survey n8003 Tshop=shoes,name:source=survey
""" """
Then place contains exactly Then place contains exactly
| object | class | extratags | | object | class | name | extratags |
| N8001 | shop | 'xx': 'yy' | | N8001 | shop | - | 'xx': 'yy' |
| N8002 | shop | 'ele': '234' | | N8002 | shop | - | 'ele': '234' |
| N8003 | shop | - | | N8003 | shop | - | - |
Scenario: Admin levels Scenario: Admin levels

5
test/bdd/steps/nominatim_environment.py
View File

@@ -89,11 +89,6 @@ class NominatimEnvironment:
if self.db_pass: if self.db_pass:
dsn += ';password=' + self.db_pass dsn += ';password=' + self.db_pass
if self.website_dir is not None \
and self.test_env is not None \
and dsn == self.test_env['NOMINATIM_DATABASE_DSN']:
return # environment already set uo
self.test_env = dict(self.default_config) self.test_env = dict(self.default_config)
self.test_env['NOMINATIM_DATABASE_DSN'] = dsn self.test_env['NOMINATIM_DATABASE_DSN'] = dsn
self.test_env['NOMINATIM_LANGUAGES'] = 'en,de,fr,ja' self.test_env['NOMINATIM_LANGUAGES'] = 'en,de,fr,ja'

9
test/bdd/steps/steps_osm_data.py
View File

@@ -49,6 +49,15 @@ def write_opl_file(opl, grid):
return fd.name return fd.name
@given('the lua style file')
def lua_style_file(context):
""" Define a custom style file to use for the import.
"""
style = Path(context.nominatim.website_dir.name) / 'custom.lua'
style.write_text(context.text)
context.nominatim.test_env['NOMINATIM_IMPORT_STYLE'] = str(style)
@given(u'the ([0-9.]+ )?grid(?: with origin (?P<origin>.*))?') @given(u'the ([0-9.]+ )?grid(?: with origin (?P<origin>.*))?')
def define_node_grid(context, grid_step, origin): def define_node_grid(context, grid_step, origin):
""" """

2
test/python/config/test_config.py
View File

@@ -222,7 +222,7 @@ def test_get_import_style_intern(make_config, src_dir, monkeypatch):
monkeypatch.setenv('NOMINATIM_IMPORT_STYLE', 'street') monkeypatch.setenv('NOMINATIM_IMPORT_STYLE', 'street')
expected = src_dir / 'settings' / 'import-street.style' expected = src_dir / 'settings' / 'import-street.lua'
assert config.get_import_style_file() == expected assert config.get_import_style_file() == expected