OpenStreetMap

OpenStreetMap + Wikidata

Posted by rtnf on 10 January 2024 in English.

So, one day, I stumbled across US State Boundary QA Checks, a web app utility for identifying issues with boundary relations in the US by utilizing both OSM and Wikidata. This app queries both Wikidata (via SPARQL Query Service) and OpenStreetMap data (via Overpass API) by using simple Javascript.

For a long time, I’ve been quite interested in the concept of “integrating both Wikidata and OpenStreetMap data, then using the coalesced data for domain-specific purposes,” but I have never done it before. Perhaps I can learn something from this source code on how to query both of them and merge the data (by using Javascript).

Here’s what I found.

Querying Overpass
function stringifyParams(params) {
    return Object.keys(params)
        .map(key => `${encodeURIComponent(key)}=${encodeURIComponent(params[key])}`)
        .join('&');
}

var overpassUrl = 'http://overpass-api.de/api/interpreter'
var relationID = 2388361 + 3600000000;
var query2 = `[timeout:180][out:csv(::id,wikidata,wikipedia,admin_level,boundary,name,"name:en";true;',')];
        area(id:${relationID})->.a;
        (
          rel[boundary=administrative][admin_level~"^7|8|9$"](area.a);
        );
        out;`;
var xhr2 = new XMLHttpRequest();
xhr2.open('POST', overpassUrl, true);

xhr2.onreadystatechange = function () {
           if (xhr2.readyState === 4) { // Check if the request is complete
               if (xhr2.status === 200) { // Check if the request was successful
                   console.log('Response:', xhr2.responseText);
               } else {
                   console.error('Error:', xhr2.status, xhr2.statusText);
               }
           }
       };

xhr2.send(stringifyParams({data:query2}))

Note :

  • Overpass Area ID is OpenStreetMap Relation Id + 3600000000
Querying Wikidata
var query = `SELECT ?item ?itemLabel WHERE 
{
  ?item wdt:P31 wd:Q146. 
  SERVICE wikibase:label { bd:serviceParam wikibase:language "en". } 
}`;
var abc = encodeURIComponent(query)
var uri = "https://query.wikidata.org/sparql?format=json&query=" + abc
var xhr = new XMLHttpRequest();
xhr.onreadystatechange = function() {
    if (xhr.readyState == 4 && xhr.status == 200) {
        var data = JSON.parse(xhr.responseText);
        console.log(data);
    }
    else{
        console.log("error")
    }
};
xhr.send();

Well, currently, I don’t have any project ideas regarding OSM + Wikidata integration. But I’ll definitely save these code blocks for future reference, just in case I need them someday.

Discussion

Comment from Minh Nguyen on 13 January 2024 at 21:58

Have you checked out QLever yet? It’s a fast alternative for federated queries on the server side. This diary post provides some examples to work off of.

Comment from rtnf on 13 January 2024 at 22:09

Yes I have.

The only concern that I have with QLever is that there’s a lag between OSM’s latest data update and when that new data can finally be queried. (It doesn’t even fetch the latest version of the Wikidata knowledge base. Instead, it creates a local periodic copy and then performs fast queries based on that periodic snapshot.)

I’m not sure how long this lag is, but that’s why I’m still sticking to the Overpass API, at least for now.

Comment from rtnf on 13 January 2024 at 22:12

Comment from Minh Nguyen on 13 January 2024 at 22:14

I see. The possibility of minutely updates was one of the nice things about Sophox back when it was functional. It also queried Wikidata directly instead of keeping a local copy, at the expense of running time.

Comment from Minh Nguyen on 15 January 2024 at 20:01

The possibility of minutely updates was one of the nice things about Sophox back when it was functional.

Sophox is functional again!

Log in to leave a comment