OpenStreetMap (OSM) is fun and fantastic for so many reasons. One being able to roam around the world from the comfort of your chair. I started mapping on OpenStreetMap in 2015 by coincidence through a friend. And OSM later became part of my work at Mapbox. Where I went through out the world as an armchair mapper. I used to maintain a list of interesting places as I encountered them. Then map it later when I am free. Here are my 3 memorable OpenStreetMap edits.

Valparai, Tamil Nadu, India

Tea plantations in Valparai by Thangaraj Kumaravel | Image license: CC BY 2.0 Deed

I have been to Valparai several times as a kid during summer vacations. It’s surreal and beautiful. My uncle’s place there used to be at the top of the hill, above the clouds itself. You won’t be able to see the road when it is cloudy. We used to play hide and seek in these gigantic guava trees. Or maybe they felt gigantic as I was a kid then. Trees were our targets because leopards were regular visitors and we were not “technically” allowed to go out of house plot.

Valparai hence was one of the places I always wanted to map due to cherished memories. Which is what I did. I have put in hours here to map detailed map features. Rekindling the memories while doing so were worth the time I have put into adding data here.

Screenshot of Valparai in openstreetmap.org

My edit

Screenshot of my edits on Valparai in openstreetmap.org

Pettineo, Sicilia, Italy

Image of Pettineo in Wikipedia Image details: © LigaDue | License: CC BY-SA 3.0 Deed

I came across Pettineo, Sicilia thanks to end user feedback from Mapbox maps. Amazing place. At least from the satellite imagery, it was. To my surprise, it was not mapped at all when I looked at it in JOSM. The edit I did here was far far more grander than Valparai. Or that’s how I remember it. What I didn’t know was that JOSM somehow hadn’t loaded the data of the place. Almost 4ish hours or so of my time resulting in duplicated data as I realised I have mapped over the original data. My edit was extremely detailed over the original data as I recall it. But it did not connected to any data that was already present. Merging felt far more complex and I had to revert my edit.

After the initial panic and changeset revert, I did edit Pettineo again, it was not as large as my first edit since I didn’t have to map half of it. I was just relieved that I was able to fix it as soon as I messed up.

Screenshot of Pettineo in openstreetmap.org

My edit

Screenshot of my edits on Pettineo in openstreetmap.org

Dierks Lake, Arkansas

Dierks Lake, Arkansas, USA Image details: © Recreation.gov

Over the years, I have created a huge interest in natural features. Especially water bodies. It started with one of my first few edits where I added some of the swimming pools in Delhi which hosted most of the foreign embassies in India.

Dierks Lake, Arkansas came through work as well. It had only extremely bare geometry. Which I hate in a waterbody. Or anything natural. Whether it is a waterbody or a tree, they have distinct shape and geometry which should be celebrated. Editing the entire lake was around ~2ish hours of work IIRC. Dierks Lake’s mapping was the reason I became a fan of waterbody mapping. And hence has a sweet spot in my heart.

My edit

My edit on Dierks Lake, Arkansas, USA ## What’s next? Once you have maps in your life, you will see it everywhere. Apart from some of the map related projects I have been working on, I wanna restart mapping in OSM. Especially natural features like forests, water bodies etc. Here is to creating more map memories.

At Mapbox, we have been looking to improve the coverage of road network in India, which is among the lowest in the world on OSM in comparison to CIA world factbook. We used the osm-coverage tile reduce processor to get a more granular breakdown of highway coverage per state. Comparing these numbers to those officially reported by the Ministry of Road Transport and Highway (MORTH) in India allowed us to better understand gaps in the coverage.

We used GEOnet Name Server’s (GNS) NGA data for the names of towns and villages which were missing in OSM and visualised these as WMS layer in JOSM. It helped us in figuring out the via points a national highway is passing and also in adding missing towns and villages.

After that, we extracted states that have the [lowest national highway coverage] (https://docs.google.com/spreadsheets/d/1uYmQEyZqP2tMe1cJYJUVcNIhFPCdzETsBh1kaSCZplI/edit#gid=818481766&vpid=A1) so that we can start to add/improve the roads one state at a time. We used road classification and tagging scheme for India as our guide for tagging the roads.

Improving highway coverage for a state:

We started with the state of Manipur which had only 67.98% coverage of the trunk road network (National Highways) in OSM. We were able to come up with a base workflow which could help anyone who is interested in improving the road network in India.

• First step was to check whether the national highways were already marked in OSM. In order to check this, we used the statistics from MORTH and cross checked the same with the current OSM data.
• Searching for the cities or towns through which a national highway passes was the next step. We used MORTH data for this.
• For the cities and towns which are not marked in OSM, we used GNS’s NGA layer to get the approximate locations of the cities/towns. This helped us determine the via route, starting and ending point of a highway. To add the NGA layer follow this post

NGA layer in action

• Finally we checked if a national highway is marked with highway tags, names, and have relations wherever needed. If this was missing, we added names, relations or retagged the roads appropriately.

How to create a relation for multiple highways & Tags to be added while creating a relation

Challenges:

• Since the names of the national highway is changing from time to time, it was difficult for us to determine whether the data was up to date or not in OSM.
• The towns and villages which were not marked in OSM made it difficult to get the exact starting and ending points for a National highway even with the data available from MORTH.

We invite the OpenStreetMap India community to help us improve road network coverage. Get in touch if you have questions and you can learn more about this project from our mapping ticket.

The past 2 months my team at Mapbox have been working full-time on one of our largest data projects to date - Realigning the highways of Japan on OSM, which is now reaching its final stages.

Japan looks thoroughly mapped on OSM and is dense with highways and local streets. On inspecting the data however, we found that most of the roads do not match the underlying satellite imagery or GPS tracks.

To complicate matters, we found that the Bing imagery has strange distortions and is poorly orthorectified and offset from GPS tracks. Inititally we thought this could be an isolated case, but on comparing with StravaGlobalheat layer these distortions continue throughout the country in an inconsistent manner.

Comparison of Bing and ortho imagery from Geospatial Information Authority of Japan (GSI)

Further research revealed that most of the map data comes from a poorly documented import from Yahoo Japan in 2011 and the the OSM-JP community has been struggling to to execute a post import cleanup of the massive dataset. A complete list of data issues we found has been now documented in this diary post. Our finding was that most of the data has not been touched since the import and remains to be of questionable quality.

This motivated us to kick into action and see how our team resources could be used to help out in such a situation.

Confirming findings

The first step was reaching out to the OSM-JP community to make sure our findings were on the right track. The most accessible point of contact was Taichi-san, a prolific mapper and Director of OpenStreetMap Foundation Japan. Our full investigation and workflow were documented on our public /mapping repository and our findings were confirmed by Taichi - the imported data in Japan had serious data issues.

Thanks to the community we found the availability of orthorectified satellite imagery from GSI which perfectly matches GPS data and is the only reliable imagery source for the country. The coverage of the imagery includes most urban areas but is not complete for Japan.

Coverage areas for GSI imagery compared with density of geolocated tweets

Moving the map

Realigning existing road networks on such a large scale is a tough problem similar to that of the US TIGER import which has still not been completely fixed after 8 years. Unlike in the US where it was possible to compare later TIGER datasets with OSM to help find differences to cleanup, there was no way to automatically calculate the scale of the issue in Japan, which meant a completely manual cleanup by eyeballing every road. That amounts to over 5 million roads in the import. The current set of mapping tools have been built for data creation and addition rather than rework. To establish a workflow, we did a trial run in a less dense part of northern Japan using the tasking manager to evolve a process for the cleanup. Some of our observations:

• GSI imagery coverage was limited to parts of Japan and we would need to focus on these areas first
• Road were split into small segments which needed to be merged for easy realignment
• Divided highways were traced as a single road instead of parallel oneway roads
• Unwanted tags like yh-width where present from the import and had to be discarded for easy merging
• Prioritizing the remapping by focusing on only major roads (motorway, trunk, primary, secondary and tertiary) was more practical than realigning all the roads which would be a never ending task.
• Realigning data in multiple passes, each limited on one class of roads helps maintain focus on a task

After a rough idea on the remapping process, we created a series of projects with comprehensive instructions on the teachosm tasking manager and spread the word on the OSM Diaries and the talk-ja mailing list to encourage more community participation in the effort.

Remapping project areas and statistics

### Tasks on http://tasks.teachosm.org

Links: Nagoya Mega task

Tokyo Mega task

Tasks on http://tasks.openstreetmap.in

Links: Kyushu

Osaka

Mito

Fukushima

Ishikawa

Beppu-Oita

Aomori

North Mutsu

Note A pass is a mapping run for a task area. Pass 1 = fixing motorway, trunk, primary, secondary roads. Pass 2 = fixing tertiary roads. Pass 3 = fixing unclassified and residential roads

To scale up the merging process, @Rub21 made a script which smartly merges highways taking into account the tags (to merge roads with identical tags) and the angle between ways (to avoid merging streets meeting at intersections). We used the whole project as a chance to discover possible tools that can be handy and speed up the mapping skills.

Outreach and community involvement

Carrying out such a large scale map improvement could not have been possible without the support of the JP community. Regular OSM dairies from our team were helpfully translated into Japanese by volunteers from talk-ja. The project instructions were also simplified and translated for the convenience of local mappers interested in joining. A Facebook Group helped gain more visibility as it tends to be a more popular medium for communicating with Japanese mappers than the mailing list.

Students of Taichi-san’s class involved in the remapping projects

In the meantime we are looking at translating our mapping guides into Japanese to allow more local mappers to join the projects using JOSM.

SOTM-Japan

Also timely was State of the Map-Japan that Mapbox got an opportunity to sponsor, and a vacationing @planemad gave the closing keynote to bring attention to the data issues and the remapping.

@planemad at SOTM Japan

The road ahead

The scope of our team’s realignment efforts was limited to only major highways (motorways, trunk, primary and secondary roads), and further by the availability of ortho imagery from GSI. The target was to correct the roads which carry the bulk of the road traffic and further continuously improve areas where we detect active map users and map feedback. We have currently covered 90% of our target improvement area and will be evaluating the impact of the cleanup in the coming weeks and building missing documentation, tools and JOSM plugins for the benefit of new mappers.

Correcting the entire street map data in Japan will take many more years and can only be accomplished by the presence of an active local community that can maintain and enrich the data using field knowledge. Meanwhile, this effort needs to serve as a reminder on the issues of large data imports and the impact it has on dynamics of community growth and participation. It also exposes how inspite of being united by a common map canvas, language still plays a significant barrier in how we communicate and support each other in OSM.

Dairy entries on remapping

Status of remapping: Light Green = Done (Pass 1), Dark Green = Done (Pass 1 and 2), Red = Not done

Some Statistics

• Area of Japan: 377,000 km2
• Area covered by GSI imagery: 180,000 km2 (47%)
• Priority project area: 120,000 (60%) km2
• Outside project area: 60,000 km2
• Cleanup progress as on 11/24: 106,000 km2 (90% of priority area)
• Data team strength: 19 members
• Number of weeks spent: 9

Here is a Map showing all the roads align/merged during this project by data team since September.

Next Steps

• As we wind down with Japan, it would be great to hear from the Japanese OSM community on the impact of this exercise. We invite Taichi-san and other members of OSM-JP to assess the work and provide us valuable feedback that can be used to improve how the Mapbox data team can help strengthen local mapping communities.
• The Japanese community can greatly benefit with localized mapping guides to use JOSM for cleanup. Interested translators can get in touch here
• If you have an idea for a new tool or JOSM plugin to help in such map cleanup tasks, propose an idea in our /mapping repository.
• Do you consider yourself an expert mapper? jump right into the latest project in Hiroshima