Excited to be attending and presenting at the State of the Asia - 2017 conference for the first time at Nepal. I will be talking on Sunday (24th September) about how we are building multilingual map of the world by linking OpenStreetMap with Wikidata - an amazing open data project by Wikimedia foundation. If you are interested about this, attend the talk in Banyan hall at 11:15 AM on Sunday, 24th.

Arabic map of the world. (View full screen map)

An interactive map built to live query the translations in Wikidata and to compare the similar sounding translations from various languages

The OpenStreetMap community has been adding wikidata tags to OpenStreetMap features for the last several months. Linking OpenStreetMap features to Wikidata can leverage both projects to create novel applications such as multi-lingual maps and integrate other semantic information which OpenStreetMap doesn’t have but Wikidata provides.

Till date, over 400K features have a wikidata tag. The integration of linking wikidata within iD and JOSM editors allowed mappers to efficiently map wikidata tags but the biggest challenge is to focus on the best match for cases when location or names do not exactly match.

This post will walk you through using wikimama, a tool to find and add missing wikidata tags in OpenStreetMap.

Matching Wikidata tags in neighborhoods with ‘wikimama’

Let’s say you want to get a list of all neighborhoods/suburbs within Stuttgart that don’t have a wikidata tag and get a list of wikidata tag matches. The wikimama tool gives you this list that you can manually review and later add to OpenStreetMap.

• Get the information of Stuttgart in OpenStreetMap

Stuttgart boundary in OpenStreetMap

We need the following information about Stuttgart: name, center longitude, center latitude, wikidata. Most cities in OpenStreetMap already have a wikidata tag, in case it doesn’t exist, you can find it directly in wikidata.org (here’s the wikidata entry for Stuttgart).

The boundary shows a radius of about 10 km will cover all neighborhoods/suburbs within Stuttgart.

• Clone the wikimama repository from Github.

• Prepare an input file to initialize wikimama. The input file needs the following:City, longitude, latitude, wikidata_id, radius, lookupdistance.

• Finally, run node batch-match.js --file input.csv.

• Once finished, wikimama will produce an output.csv listing all neighborhoods/suburbs within a 10km radius of Stuttgart and the probable wikidata tag that matches each place.

• Import the csv to a spreadsheet so you can do basic filtering and sorting. Use the JOSM url field to directly edit the feature in JOSM.

Output imported to a spreadsheet with basic filtering and formatting

What’s under the hood?

Wikimama queries OpenStreetMap for all neighborhoods/suburbs within your defined threshold (10 km). This list of queried places are then compared to existing wikidata entries under the human settlement class/subclass. The matching compares both the geographic and Levenshtein distances for each entry and gives it a score of similarity. Higher score means a good match. Full description of the output csv is available in the repo’s wiki.

To be clear, we are not proposing any automated upload of wikimama generated results to OpenStreetMap, Wikmama does not guarantee a 100% match for all queried places. This tool only provides a list of potential matches and scoring. The mapper should ultimately decide whether the match is correct and then upload to OpenStreetMap.

Our team at Mapbox have started using this tool to match wikidata tags in 20 cities and have linked for about 1,500 neighborhoods. If you need help in using wikimama or want a list of matches for your city, please comment here or directly in the wikimama repo.

A previous diary post-The invalid areas of the map introduced us to invalid relations formed due to multipolygon relation being broken due to various errors. The post has a comprehensive discussions on different errors, its severity and respective simple fixes. It also has these interesting open questions which needs to be debated.

It seems like a huge majority of the issues need to be carefully reviewed by hand and cleaned up. What does a realistic approach to clean the map look like?

The flexibility of the current map editors allows mappers to continue to create features that don’t make sense like a way tagged as a forest. Is it time for stricter validity checks on uploads?

The current diary post walks through the tools which aids in fixing these invalid multi polygon relations which not only remains erroneous part of the map but also affects the map rendering.

Start by reading this documentation on multipolygons if you are trying to understand the basics.

How to identify invalid multipolygons ?

An error debugging tool from Geofabrics - OSM Inspector tool aids in identifying these invalid relations in OpenStreetMap. The tool can be either used in a web browser interface or as a layer in JOSM editor.

To use OSM inspector in a web browser,

1. Go to http://tools.geofabrik.de/osmi
2. Select Areas from ‘View’ menu.
3. Left panel contains the list of all possible errors which enables user to toggle between errors.
4. The map window highlights the node causing error and allows user to pan, zoom over the map in order to inspect the affected portion of the relation.
5. The Selection panel at the right edge of the page displays the selected object and the small icons on the top leads you to map editors where one can fix the issue and also to open OpenStreetMap.

Elements in OSM Inpector tool page

To use it the tool JOSM editor,

1. Open Imagery preferences from the menu bar.
2. Find OSM Inspector: Area and activate the layer from the list of available layers or add this as a WMS layer in JOSM http://tools.geofabrik.de/osmi/view/areas/wxs?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetCapabilities.
3. Now, the layer will be listed under the imagery menu.
4. Load the layer and you will have a layer showing all the broken geometries with nodes highlighting at the region of errors and also a text label which mention the error around the node.

Once the error is rectified, make sure to check the roles and sorting order of the relation from the relation window before uploading the data to OpenStreetMap. Along with the tool, JOSM validator can be used to validate the geometry once it has been fixed.

Adding OSM Inspector layer to JOSM

This guide runs through a short tutorial on enabling the OSM Inspector ‘Areas’ layer and its usage in JOSM. Combination of both these validators results for a powerful error debugging tool to identify and fix errors in invalid relations.

Fixes by Mapbox team

The data team at Mapbox reviewed about 1024 invalid multipolygons and fixed around 70% of them with help of OSM inspector. Out of the rest, about 16% of them were unfixable issues. (Ex. Administrative boundaries of type=boundary and disputed borders; Segment missing in the relation causing ring not closed issues). 6% of them had a number of minor errors (Ex. Touching rings) which were not serious issues as it did neither harmed other data nor affect rendering of the map. Check out more at the fixing broken multipolygon issue.

There exists a huge number of broken/ invalid relations which needs to be fixed. These invalid areas of the map can only be fixed by the local community mappers. Would love to hear any suggestions or feedbacks and other tools that can be utilised to identify the broken multi polygon relations. Have any questions, leave a comment below.

Our data team at Mapbox along with the OSM-TW community recently completed mapping and aligning road network in Taiwan. The realignment was carried out mainly based on newly rendered Mapbox satellite imagery and Bing imagery.

Progress over Taiwan during road network realignment.

During this project, we observed offsets in Bing imagery in several areas when compared with Strava and GPS traces. All these offsets are now included in the Imagery Offset Database for the wider community to use.

Imagery offsets in Taiwan.

For details on how to use the Imagery Offset Database in JOSM, please read @krisna’s diary post. For any feedback/suggestions, please comment below.

A review of the existing data in 5 most populated cities of France was detailed in our previous diary. Through this diary, we will be introducing you all to a handy tool for mapping turn restrictions.

Adding turn restrictions using OSM Navigation Map

OSM navigation map is an open tool that was built to review and add missing data for turn restrictions. It relies on Mapillary as a source to identify the turn restrictions in the detected traffic signages from the street level images captured by the volunteers.

The data team in Mapbox has used this tool for reviewing the existing restrictions and as well adding the missing ones for US cities, 5 cities each in Canada and Germany. The tool assists in reviewing the restrictions easily and quickly.

Detected signage of only_right_turn restriction in Mapillary

Detected signage of no_right_turn restriction in Mapillary

Verifying turn restriction with images from Mapillary in OSM Navigation Map

Reviewing turn restriction in OSM Navigation Map

Explore the tool with a detailed guide on GitHub on how to use the tool for reviewing and adding missing turn restrictions on OpenStreetMap.

Community participation has already ensured that around 436 turn restrictions across 5 cities (Paris, Marseille, Lyon, Toulouse, and Nice) of France :clap: :tada: . Along with the local knowledge, this tool would be a great companion to improve data in OpenStreetMap for navigation.

If it’s your first time mapping turn restrictions, read the guide on mapping turn restrictions using Mapillary to understand different scenarios and special cases. Help us by updating the special cases list for France that would help all community mappers as well our mapping team to be aware of these cases while reviewing turn restrictions.

We would love to collaborate with the community on improving the data in OpenStreetMap by collectively getting feedback on our tools as well as workflow.

As most of us in the team are English speakers, it is difficult for us to translate the guide into French. It would be great if the community members can help us in translating the OSM Navigation Map guide to French and host it on OpenStreetMap wiki or anywhere that would be valuable for local mappers to refer.

Merci.

One of the prime applications of maps is navigation. In order to make OpenStreetMap more reliable for navigation, the team Mapbox has been improving the data in the map by adding missing turn restrictions, exits and destinations across the world. Recently, we mapped most of the gaps in the data across US cities as well as 5 cities each in Canada and Germany. In the entire process, we worked closely with the local community by taking inputs in the form of suggestions and guidance on improving our workflow and resources.

Continuing from our previous work in Europe, we did an initial research on France to get estimate of existing navigation data. These are the 5 cities we want to start with,

Our initial assessment shows that there are very few turn-restrictions mapped overall, and a majority of what is present is “only-straight” or “only-right” rather than the common “no-right-turn” or “no-left-turn”.

Data Sources for Mapping

Mapillary Imagery

We use Mapillary as the primary source for adding turn restrictions. Overall, Paris has the best Mapillary coverage which would allow for bridging any gaps in data. This is followed by Lyon, where coverage is less dense, but could still be used to cover some gaps, if any.

We came across these situations while reviewing the data using Mapillary images, where we were unable to perceive these signage representations.

[1] Mapping conditional turn restrictions

There is a no left turn but with a condition for except service vehicles. But it is only mapped as no-left-turn. How should we be mapping this?

[2] Similar to the above case, is this sign a combination of all the three or are they individual signs? How do we interpret sign like these in general?

[3] Exit numbers

There seems to be signs with only_arrows along with an E below it. They are mostly located on higher classification of roads. What do these signs signify?

http://mapbox.github.io/osm-navigation-map/#19.11/48.90057/2.35856

[4] We encountered multiple restriction signs, What is the background behind these signs and what is the ideal way to handle these?

Although most of these turn restrictions are mapped, there are still many conditional turn restrictions that could be added and a few existing turn restrictions that could be validated for improving data quality on OpenStreetMap. It would be great if community can answer our questions and help us understand the above scenarios better. Looking forward to hear your suggestions and comments on this.

We’re going to be at SOTM in Brussels next week. Catch up with @jinalfoflia @ramyaragupathy @pratikyadav @geohacker and @planemad on the latest data team projects at Mapbox!

Thank you.

Field mapping!

Though we have high resolution satellite imageries and robust tools to digitise a map, we end up not tracing many features due to lack of visibility or some errors or covers. The only solution if an area has to be mapped completely and precisely is field mapping.

Last week a team of 6 involving Srividya, Krishna, Maanya, Amisha, Upendra and I decided to head out for field mapping at 12th main road in Indiranagar, Bengaluru. We aimed at collecting building details, building levels, Point of Interests (POIs), trees and street lights. We walked for more than 3km and tracked our ways using smartphone applications, mainly OSMTracker.

Street view images from Mapillary

We used Mapillary for capturing geo-tagged photos along the way and OSMTracker to collect data of the features while on the move. We added a text note at the location to feed the details of the feature such as building level, building number and names of commercial establishments. For mapping trees and street lights few of the team members used the presets available in the OSMTracker, which turned out to be handy.

Collective edits extracted from Overpass Turbo

In about two hours of time we were able to add around 151 trees, 44 street lamps and 103 buildings.

View of 12th Main on OSM after field mapping

We converted the traces recorded to .gpx format, mapped the collected data using JOSM and finally uploaded them to OpenStreetMap.

On the whole, field mapping was a wonderful experience which gave us an opportunity to explore our neighbourhood and turned out to be a great learning as well as a fun activity.

I have been mapping my hometown for quite a while now. This is where I am residing since birth and had my education in and around Gubbi. In Kannada language the term is related to a sparrow bird (ಗುಬ್ಬಚ್ಚಿ) . The town has a history of more than 150 years. This is the place from where I would love to start mapping.

About Gubbi

Gubbi is geographically about 100km to the west of Bengaluru, the capital of Karnataka. The national highway NH-73 (previously NH-206) (Also known as Bengaluru - Honnavara Road) runs along Gubbi. The population of the Gubbi Town is 18, 457 as per Census 2011. The total area of the Town is 6.67 sq. km.

Status of map before putting my hands

This was how map of Gubbi looked before I started mapping

Most of the basic amenities and road networks in and around Gubbi was already on OSM. I thank OSM community for their contributions to the map around Gubbi.

The areas I worked on

Addition of missing POIs, buildings, roads and correction of errors in existing features.

Things I have added and improved

• Missing roads, tracks and pathways
• Added my home address and few residential buildings in the neighborhoods
• Added a railway track, platform and improved map of Gubbi railway station
• Added public buildings and POIs
• Added farms and fields around Gubbi

---

Few buildings that were added

Few edits on road network

---

Here are few of my additions

Close up of my edits

---

What do I foresee?

I always loved adding spatial data on to the map and have been enjoying mapping my hometown. For now I have added few features which I am intimately aware of. My plan is to add more details based on satellite imagery and local knowledge, and do a field survey with mapillary and/or field papers and improve the locality on OpenStreetMap.

I’d also welcome any comments from the mapping community on things that needs to be improved and also on what I’ve mapped so far, or tips for mapping.

Thank you.