Data Mapping

The data mapping defines which OSM feature types should be imported in which table. The mapping is a YAML (or JSON) file.

See example-mapping.yml for an example.

Note

The YAML format uses indentations to indicate nesting. Tab characters are not allowed. Quotes around simple strings are optional with YAML. "simple_string", 'simple_string' and simple_string are all equal. However, numbers and boolean values (yes, no, true, false) need to be quoted when they should be interpreted as a string (for example, when filtering building: ['no']).

Tables

The most important part is the tables definition. Each table is a YAML object with the table name as the key. Each table has a type, a mapping definition and columns.

type

type can be point, linestring, polygon, geometry, relation and relation_member. geometry requires a special type_mappings. Relations are described in more detail here.

mapping

mapping defines which OSM key/values an element needs to have to be imported into this table. mapping is a YAML object with the OSM key as the object key and a list of all OSM values to be matched as the object value. You can use __any__ to match all values (e.g. amenity: [__any__]). To match elements regardless of their tags use __any__: [__any__]. You need to use load_all in this case so that Imposm has access to all tags.

To import all polygons with tourism=zoo, natural=wood or natural=land into the landusages table:

 tables:
   landusages:
     type: polygon
     mapping:
       natural: [wood, land]
       tourism: [zoo]
     

relation_types

relation_types restricts which relation types should be imported. It is a list with type values, e.g. [route, master_route].

For tables of type relation and relation_member: Only import relations which have this type value. You still need to have a mapping.

For tables of type polygon: Only build multi-polygons for relations which have this type value. You still need to have a mapping. Defaults to [multipolygon, boundary, land_area].

tables:
  routes:
    type: relation
    relation_types: [route]
    mapping:
      route: [bus]

columns

columns is a list of columns that Imposm should create for this table. Each column is a YAML object with a type and a name and optionally key, args and from_member. The legacy name of columns parameter is fields, but it should not be used. If both columns and fields exist in the same mapping, fields will be used. Support for ``fields` will be removed in the future versions.

name

This is the name of the resulting column.

type

This defines the data type that Imposm should use for this column. There are two different classes of types. Value types are types that convert OSM tag values to a specific database type. Examples are string for street or place names, or bool for yes, no, true or false values. Element types are types that dependent on the OSM element (node/way/relation). Examples are geometry for the geometry, mapping_key and mapping_value for the actual key and value that was mapped.

See Column types for documentation of all types.

key

key defines the OSM key that should be used for this column. This is required for all value types.

args

Some column types require additional arguments. Refer to the documentation of the type.

from_member

from_member is only valid for tables of the type relation_member. If this is set to true, then tags will be used from the member instead of the relation.

filters

You can limit which elements should be inserted into a table with filters. You can require specific tags or reject elements that have specific tags. require and reject accept keys and a list of values, similar to a mapping. You can use __any__ to require or reject all values (e.g. amenity: [__any__]).

require_regexp and reject_regexp can be used to filter values based on a regular expression. You can use the Go Regex Tester to test your regular expressions.

The following mapping only imports buildings with a name tag. Buildings with building=no or building=none or buildings with a non-numeric level are not imported.

tables:
  buildings:
    type: polygon
    filters:
      require:
        name: [__any__]
      reject:
        building: ['no', none]
      reject_regexp:
        level: '^\D+.*$'
    mapping:
      building: [__any__]
    columns:
      ...

Note

Regular expressions in require_regexp and reject_regexp should be enclosed in single quotes ('). Otherwise YAML will interpret backslashes as escape sequences.

Note

You can only filter tags that are referenced in the mapping or columns of any table. See Tags on how to make additional tags available for filtering.

Example

The mapping below will create a tracks table with the following columns:

  • osm_id with the ID of the way

  • the_geom with a LineString geometry

  • street_name with the content of the OSM name tag

  • is_bridge with a true value if the OSM bridge tag is true-ish (1, yes or true), otherwise it will be false

  • highway_type with the OSM value that was matched by the mapping of this table. In this example one of path, track, or classified.

tables:
  tracks:
    type: linestring
    mapping:
      highway: [path, track, unclassified]
    columns:
    - {name: osm_id, type: id}
    - {name: the_geom, type: geometry}
    - {key: name, name: street_name, type: string}
    - {key: bridge, name: is_bridge, type: bool}
    - {name: highway_type, type: mapping_value}

mappings

An OSM element is only inserted once even if a mapping matches multiple tags. Sometime it’s convenient to have a geometry multiple times, e.g. a way with rail=tram and highway=secondary. mappings allows to define multiple sub-mappings. Each sub-mapping requires a name and a separate mapping dictionary. The elements will be inserted into the table for each match of a sub-mapping.

 tables:
   transport:
     type: linestring
     mappings:
       rail:
         mapping:
           rail: [__any__]
       roads:
         mapping:
           highway: [__any__]
       

type_mappings

A table with a type of geomery can contain different geometry types, such as points, linestrings, and/or polygons. The type_mappings provides separate mapping for the required geometries types.

 tables:
   all:
     columns:
       - name: osm_id
         type: id
       - name: geometry
         type: geometry
       - name: tags
         type: hstore_tags
     type: geometry
     type_mappings:
       points:
         amenity: [__any__]
         poi: [__any__]
         shop: [__any__]
       linestrings:
         highway: [__any__]
       polygons:
         landuse: [__any__]
         poi: [__any__]
         shop: [__any__]

Column types

Value types

bool

Convert true, yes and 1 values to true, otherwise use false.

boolint

Same as bool but stores a numeric 1 for true values, and 0 otherwise.

string

The value as-is. Note that missing values will be inserted as an empty string and not as null. This allows SQL queries like column NOT IN ('a', 'b').

direction

Convert true, yes and 1 to the numeric 1, -1 values to -1 and other values to 0. This is useful for oneways where a -1 signals that a oneway goes in the opposite direction of the geometry.

integer

Convert values to an integer number. Other values will not be inserted. Useful for admin_levels for example.

enumerate

Enumerates a list of values and stores tag values as an integer.

The following enum column will contain 1 for landuse=forest, 4 for landuse=grass and 0 for undefined values.

columns:
  - name: enum
    type: enumerate
    key: landuse
    args:
        values:
           - forest
           - park
           - cemetery
           - grass

mapping_value will be used when key is not set or null.

wayzorder

Calculate the z-order of an OSM highway or railway. Returns a numeric value that represents the importance of a way where motorway is the most important (9), and path or track are least important (0). bridge and tunnel will modify the value by -10/+10. layer will be multiplied by ten and added to the value. E.g. highway=motorway, bridge=yes and layer=2 will return 39 (9+10+2*10).

You can define your own ordering by adding a list of ranks. The z-order value will be the index in the list (starting with 1). bridge, tunnel, and layer will modify the value by the number of items in the ranks list, instead of 10. Use default to set the default rank.

columns:
  - name: zorder
    type: wayzorder
    args:
        default: 5
        ranks:
           - footway
           - path
           - residential
           - primary
           - motorway

A motorway will have a zorder value of 5, a residential with bridge=yes will be 8 (3+5).

categorize

Stores a number depending on a value, similar to enumerate. However, categorize allows you to explicitly configure the number for each value, multiple values can have the same number and it can search in multiple keys. You can use this to implement a scale rank for sorting elements depending on their relative importance.

- args:
    default: 0
    values: {
      FR: 10, NL: 8, LU: 3,
    }
  keys:
  - country_code_iso3166_1_alpha_2
  - ISO3166-1:alpha2
  - ISO3166-1
  name: scalerank
  type: categorize_int

geojson_intersects and geojson_intersects_field

Checks whether the geometry of the element intersects geometries from a provided GeoJSON file. geojson_intersects returns true if it intersects any geometry. geojson_intersects_field returns a string property of the intersected feature.

- args:
    geojson: special_interest_areas.geojson
  name: in_special_interest_area
  type: geojson_intersects
- args:
    geojson: special_interest_areas.geojson
    property: area
  name: special_interest_area_name
  type: geojson_intersects_field

Element types

id

The ID of the OSM node, way or relation. Relation IDs are negated (-1234 for ID 1234) to prevent collisions with way IDs.

mapping_key

The OSM key that was matched by this table mapping (highway, building, nature, landuse, etc.).

Note

Imposm will choose the first key of the table mapping if an OSM element has multiple tags that match. For example: mapping_key will use natural for an OSM element with landuse=forest and natural=wood tags, if natural comes before landuse in the table mapping. You need to define an explicit column if you need the value of a specific tag (e.g. {“type”: “string”, “name”: “landuse”, “key”: “landuse”}).

mapping_value

The OSM value that was matched by this table mapping (primary, secondary, yes, forest, etc.).

Note

The note of mapping_key above applies to mapping_values as well.

geometry

The geometry of the OSM element.

validated_geometry

Like geometry, but the geometries will be validated and repaired when this table is used as a source for a generalized table. Must only be used for polygon tables.

area

Area of polygon geometries in the unit of the selected projection (m² or degrees²). Note that the area is only accurate at the equator for EPSG:4326 and EPSG:3857 and gets off the more the geometry moves to the poles. It’s still good enough to sort features by area for rendering purposes.

webmerc_area

Area of polygon geometries in m². This field only works for the webmercator projection (EPSG:3857). The latitude of the geometry is considered when calculating the area. This area is not precise. Polygons lower than 70° latitude should have a webmerc_area within ±20% of the true size. However, long polygons like a runway can exhibit a much larger error.

hstore_tags

Stores tags in an hstore column. Requires the PostgreSQL hstore extension. You can select tags with the include option, otherwise all tags will be inserted.

In any case, hstore_tags will only insert tags that are referenced in the mapping or columns of any table. See Tags on how to make additional tags available for import.

Element types for relation_member

The following types are only valid for tables of the type relation_member.

member_id

The OSM ID of the relation member.

member_type

The type of the relation member. 0 for nodes, 1 for ways and 2 for relations.

member_role

The role of the relation member as a string, e.g. outer, stop, etc.

member_index

The index of the member in the relation, starting from 0. E.g. the first member is 0, second member is 1, etc. This can be used to query bus stops of a route relation in the right order.

Generalized Tables

Generalized tables allow you to create a copy of an imported table with simplified/generalized geometries. You can use these generalized tables for rendering low map scales, where a high spatial resolution is not required.

Each generalize table is a YAML object with the new table name as the key. Each generalize table has a source and a tolerance and optionally an sql_filter.

source is the table name of another Imposm table from the same mapping file. You can also reference another generalized table, to create multiple generalizations of the same data.

tolerance is the resolution used for the Douglas-Peucker simplification. It has the same unit as the import -srid, i.e. meters for EPSG:3857 and degrees for EPSG:4326. Imposm uses PostGIS ST_SimplifyPreserveTopology.

The optional sql_filter can be used to limit the rows that will be generalized. You can use it to drop geometries that are to small for the target map scale.

generalized_tables:
  waterareas_gen_50:
    source: waterareas
    sql_filter: ST_Area(geometry)>50000.000000
    tolerance: 50.0

Tags

Imposm caches only tags that are required for a mapping or for any columns. This keeps the cache small as it does not store any tags that are not required for the import. You can change this if you want to import other tags, e.g with the hstore_tags column type.

Add load_all to the tags object inside your mapping file. You can still exclude tags with the exclude option. exclude supports a simple shell file name pattern matching. exclude has only effect when load_all is enabled.

Alternatively you can list all tags that you want to include with the include option. include does not support pattern matching and it has no effect when load_all is used.

To load all tags except created_by, source, and tiger:county, tiger:tlid, tiger:upload_uuid, etc:

tags:
  load_all: true,
  exclude: [created_by, source, "tiger:*"]

To load specific data about amenities for inclusion into an hstore_tags column:

tags:
  include: [operator, opening_hours, wheelchair, website, phone, cuisine]

Areas

A closed way is way where the first and last nodes are identical. These closed ways are used to represent elements like building, forest or park polygons, but they can also represent linear (non-polygon) features, like a roundabout or a race track.

OpenStreetMap uses the area tag to specify if a closed way is an area (polygon) or a linear feature (linestring). For example highway=pedestrian, area=yes is a polygon feature.

By default, Imposm inserts all closed ways into polygon tables as long as area is not no and linestring tables will contain all closed ways as long as the area is not yes. However, the area tag is missing from most OSM elements, as buildings, landuse, etc. should be interpreted as area=yes by default and highways for example are area=no by default.

You can configure these default interpretations with the areas option.

areas:
  area_tags: [building, landuse, leisure, natural, aeroway]
  linear_tags: [highway, barrier]

With this areas configuration, highway elements are only inserted into polygon tables if there is an area=yes tag. aeroway elements are only inserted into linestring tables if there is an area=no tag.