move php tests in common test dir and unify READMEs

phpunit.xml

@@ -8,13 +8,13 @@
     processIsolation="false"
     stopOnFailure="false"
     syntaxCheck="true"
-    bootstrap="tests-php/bootstrap.php"
+    bootstrap="test/php/bootstrap.php"
     >
     <php>
     </php>
     <testsuites>
         <testsuite name="Nominatim PHP Test Suite">
-            <directory>./tests-php/Nominatim</directory>
+            <directory>./test/php/Nominatim</directory>
         </testsuite>
     </testsuites>
     <filter>

test/README.md

@@ -0,0 +1,133 @@
+This directory contains functional and unit tests for the Nominatim API.
+
+Prerequisites
+=============
+
+ * Python 3 (https://www.python.org/)
+ * behave test framework >= 1.2.5 (https://github.com/behave/behave)
+ * nose (https://nose.readthedocs.org)
+ * pytidylib (http://countergram.com/open-source/pytidylib)
+ * psycopg2 (http://initd.org/psycopg/)
+
+To get the prerequisites on a a fresh Ubuntu LTS 16.04 run:
+
+     [sudo] apt-get install python3-dev python3-pip python3-psycopg2 python3-tidylib phpunit
+     pip3 install --user behave nose
+
+
+Overall structure
+=================
+
+There are two kind of tests in this test suite. There are functional tests
+which test the API interface using a BDD test framework and there are unit
+tests for specific PHP functions.
+
+This test directory is sturctured as follows:
+
+ -+-   bdd         Functional API tests
+  | \
+  | +-  steps      Step implementations for test descriptions
+  | +-  osm2pgsql  Tests for data import via osm2pgsql
+  | +-  db         Tests for internal data processing on import and update
+  | +-  api        Tests for API endpoints (search, reverse, etc.)
+  |
+  +-   php         PHP unit tests
+  +-   scenes      Geometry test data
+  +-   testdb      Base data for generating API test database
+
+
+PHP Unit Tests
+==============
+
+Unit tests can be found in the php/ directory and tests selected php functions.
+Very low coverage.
+
+To execute the test suite run
+
+   cd test/php
+   phpunit ../
+
+It will read phpunit.xml which points to the library, test path, bootstrap
+strip and set other parameters.
+
+
+BDD Functional Tests
+====================
+
+Functional tests are written as BDD instructions. For more information on
+the philosophy of BDD testing, see http://pythonhosted.org/behave/philosophy.html
+
+Usage
+-----
+
+To run the functional tests, do
+
+    cd test/bdd
+    behave
+
+The tests can be configured with a set of environment variables:
+
+ * `BUILD_DIR` - build directory of Nominatim installation to test
+ * `TEMPLATE_DB` - name of template database used as a skeleton for
+                   the test databases (db tests)
+ * `TEST_DB` - name of test database (db tests)
+ * `ABI_TEST_DB` - name of the database containing the API test data (api tests)
+ * `TEST_SETTINGS_TEMPLATE` - file to write temporary Nominatim settings to
+ * `REMOVE_TEMPLATE` - if true, the template database will not be reused during
+                       the next run. Reusing the base templates speeds up tests
+                       considerably but might lead to outdated errors for some
+                       changes in the database layout.
+ * `KEEP_TEST_DB` - if true, the test database will not be dropped after a test
+                    is finished. Should only be used if one single scenario is
+                    run, otherwise the result is undefined.
+
+Logging can be defined through command line parameters of behave itself. Check
+out `behave --help` for details. Also keep an eye out for the 'work-in-progress'
+feature of behave which comes in handy when writing new tests.
+
+Writing Tests
+-------------
+
+The following explanation assume that the reader is familiar with the BDD
+notations of features, scenarios and steps.
+
+All possible steps can be found in the `steps` directory and should ideally
+be documented.
+
+### API Tests (`test/bdd/api`)
+
+These tests are meant to test the different API endpoints and their parameters.
+They require a preimported test database, which consists of the import of a
+planet extract. The polygons defining the extract can be found in the test/testdb
+directory. There is also a reduced set of wikipedia data for this extract,
+which you need to import as well.
+
+The official test dataset is derived from the 160725 planet. Newer
+planets are likely to work as well but you may see isolated test
+failures where the data has changed. To recreate the input data
+for the test database run:
+
+    wget http://free.nchc.org.tw/osm.planet/pbf/planet-160725.osm.pbf
+    osmconvert planet-160725.osm.pbf -B=test/testdb/testdb.polys -o=testdb.pbf
+
+Before importing make sure to add the following to your local settings:
+
+    @define('CONST_Database_DSN', 'pgsql://@/test_api_nominatim');
+    @define('CONST_Wikipedia_Data_Path', CONST_BasePath.'/test/testdb');
+
+### Indexing Tests (`test/bdd/db`)
+
+These tests check the import and update of the Nominatim database. They do not
+test the correctness of osm2pgsql. Each test will write some data into the `place`
+table (and optionally `the planet_osm_*` tables if required) and then run
+Nominatim's processing functions on that.
+
+These tests need to create their own test databases. By default they will be
+called `test_template_nominatim` and `test_nominatim`. Names can be changed with
+the environment variables `TEMPLATE_DB` and `TEST_DB`. The user running the tests
+needs superuser rights for postgres.
+
+### Import Tests (`test/bdd/osm2pgsql`)
+
+These tests check that data is imported correctly into the place table. They
+use the same template database as the Indexing tests, so the same remarks apply.

test/bdd/db/import/parenting.feature

@@ -18,10 +18,14 @@ Feature: Parenting of objects
          | object | parent_place_id |
          | N1     | W1 |
          | N2     | W1 |
-        And search_name contains
+        When searching for "4 galoo"
-         | object | nameaddress_vector |
+        Then results contain
-         | N1     | 4, galoo, 12345 |
+         | ID | osm_type | osm_id | langaddress
-         | N2     | 5, galoo, 99999 |
+         | 0  | N        | 1      | 4, galoo, 12345
+        When searching for "5 galoo"
+        Then results contain
+         | ID | osm_type | osm_id | langaddress
+         | 0  | N        | 2      | 5, galoo, 99999
 
     Scenario: Address without tags, closest street
         Given the scene roads-with-pois

test/bdd/environment.py

@@ -10,7 +10,6 @@ from sys import version_info as python_version
 logger = logging.getLogger(__name__)
 
 userconfig = {
-    'BASEURL' : 'http://localhost/nominatim',
     'BUILDDIR' : os.path.join(os.path.split(__file__)[0], "../../build"),
     'REMOVE_TEMPLATE' : False,
     'KEEP_TEST_DB' : False,

test/bdd/steps/queries.py

@@ -264,7 +264,7 @@ def send_api_query(endpoint, params, fmt, context):
             for h in context.table.headings:
                 params[h] = context.table[0][h]
 
-    env = BASE_SERVER_ENV
+    env = dict(BASE_SERVER_ENV)
     env['QUERY_STRING'] = urlencode(params)
 
     env['SCRIPT_NAME'] = '/%s.php' % endpoint

test/php/Nominatim/NominatimTest.php

@@ -2,7 +2,7 @@
 
 namespace Nominatim;
 
-require '../lib/lib.php';
+require '../../lib/lib.php';
 
 class NominatimTest extends \PHPUnit_Framework_TestCase
 {

test/testdb/README.md

@@ -1,15 +0,0 @@
-Creating the test database
-==========================
-
-The official test dataset is derived from the 160725 planet. Newer
-planets are likely to work as well but you may see isolated test
-failures where the data has changed. To recreate the input data
-for the test database run:
-
-    wget http://free.nchc.org.tw/osm.planet/pbf/planet-160725.osm.pbf
-    osmconvert planet-160725.osm.pbf -B=testdb.polys -o=testdb.pbf
-
-Before importing make sure to add the following to your local settings:
-
-    @define('CONST_Database_DSN', 'pgsql://@/test_api_nominatim');
-    @define('CONST_Wikipedia_Data_Path', CONST_BasePath.'/test/testdb');

tests-php/README.txt

@@ -1,14 +0,0 @@
-Basic unit tests of PHP code. Very low coverage. Doesn't cover interaction
-with the webserver/HTTP or database (yet).
-
-You need to have
-https://phpunit.de/manual/4.2/en/
-installed.
-
-To execute the test suite run
-$ cd tests-php
-$ phpunit ./
-
-It will read phpunit.xml which points to the library, test path, bootstrap
-strip and set other parameters.
-

tests/README.md

@@ -1,102 +0,0 @@
-This directory contains functional tests for the Nominatim API,
-for the import/update from osm files and for indexing.
-
-The tests use the lettuce framework (http://lettuce.it/) and
-nose (https://nose.readthedocs.org). API tests are meant to be run
-against a Nominatim installation with a complete planet-wide
-setup based on a fairly recent planet. If you only have an
-excerpt, some of the API tests may fail. Database tests can be
-run without having a database installed.
-
-Prerequisites
-=============
-
- * lettuce framework (http://lettuce.it/)
- * nose (https://nose.readthedocs.org)
- * pytidylib (http://countergram.com/open-source/pytidylib)
- * haversine (https://github.com/mapado/haversine)
- * shapely (https://github.com/Toblerity/Shapely)
-
-Usage
-=====
-
- * get prerequisites
- 
-     # on a fresh Ubuntu LTS 14.04 you'll also need these system-wide packages
-     [sudo] apt-get install python-dev python-pip python-Levenshtein tidy
-
-     [sudo] pip install lettuce nose pytidylib haversine psycopg2 shapely
-
- * run the tests
-
-     NOMINATIM_SERVER=http://your.nominatim.instance/ lettuce features
-
-The tests can be configured with a set of environment variables:
-
- * `NOMINATIM_SERVER` - URL of the nominatim instance (API tests)
- * `NOMINATIM_DIR` - source directory of Nominatim (import tests)
- * `TEMPLATE_DB` - name of template database used as a skeleton for
-                   the test databases (db tests)
- * `TEST_DB` - name of test database (db tests)
- * `NOMINATIM_SETTINGS` - file to write temporary Nominatim settings to (db tests)
- * `NOMINATIM_REUSE_TEMPLATE` - if defined, the template database will not be
-                                deleted after the test runs and reused during
-                                the next run. This speeds up tests considerably
-                                but might lead to outdated errors for some
-                                changes in the database layout.
- * `NOMINATIM_KEEP_SCENARIO_DB` - if defined, the test database will not be
-                                  dropped after a test is finished. Should
-                                  only be used if one single scenario is run,
-                                  otherwise the result is undefined.
- * `LOGLEVEL` - set to 'debug' to get more verbose output (only works properly
-                when output to a logfile is configured)
- * `LOGFILE` - sends debug output to the given file
-
-Writing Tests
-=============
-
-The following explanation assume that the reader is familiar with the lettuce
-notations of features, scenarios and steps.
-
-All possible steps can be found in the `steps` directory and should ideally
-be documented.
-
-
-API Tests (`features/api`)
---------------------------
-
-These tests are meant to test the different API calls and their parameters.
-
-There are two kind of steps defined for these tests: 
-request setup steps (see `steps/api_setup.py`) 
-and steps for checking results (see `steps/api_result.py`).
-
-Each scenario follows this simple sequence of steps:
-
-  1. One or more steps to define parameters and HTTP headers of the request.
-     These are cumulative, so you can use multiple steps.
-  2. A single step to call the API. This sends a HTTP request to the configured
-     server and collects the answer. The cached parameters will be deleted,
-     to ensure that the setup works properly with scenario outlines.
-  3. As many result checks as necessary. The result remains cached, so that
-     multiple tests can be added here.
-
-Indexing Tests (`features/db`)
----------------------------------------------------
-
-These tests check the import and update of the Nominatim database. They do not
-test the correctness of osm2pgsql. Each test will write some data into the `place`
-table (and optionally `the planet_osm_*` tables if required) and then run
-Nominatim's processing functions on that.
-
-These tests need to create their own test databases. By default they will be
-called `test_template_nominatim` and `test_nominatim`. Names can be changed with
-the environment variables `TEMPLATE_DB` and `TEST_DB`. The user running the tests
-needs superuser rights for postgres.
-
-
-Import Tests (`features/osm2pgsql`)
------------------------------------
-
-These tests check that data is imported correctly into the place table. They
-use the same template database as the Indexing tests, so the same remarks apply.