Frankly, I don't give a flying F about the license of my edits, as long as anyone can use them.

(I DO care about the fact that a lot of existing data from others may be removed in the future, which is by itself reason enough to consider a license change plain dumb.)

I know there was an option to grant your contributions public domain status, which I selected, but that was no more than a poll. What was missing was an option to explicitly license contributions under both new and old license.

This way, in case of an OSM fork, my new edits can still be used for both.

Keepright was showing a whole bunch of streets in a residential area as unconnected. One junction point was missing, easy to fix, right?

But looking at the aerials, the real streets didn't quite match up with reality. TIGER...

Anyway, they matched up close enough to make the usual educated guesses, but since this area is really close to the Shoreline theatre, why not do a quick drive-by on my way to a movie?

Turns out: there's nothing left! What must have been cheap army housing is gone. Roads, houses, everything completely leveled!

After making all the fixes to OSM, I couldn't help checking out if the others got it right too. Bing still shows the old ways, Yahoo and Mapquest too.

But Google surely has to be up to date on that one, right? After all, the Google campus is less than half a mile away. Well, no, they too still show the old neighborhood in all its bygone glory.

I wonder how long it will take for them to catch up. (Do they read these diaries?)

If you don't want to go though the many hoops that I've described earlier to get USGS topo's as background in Potlatch or Merkaartor and you don't want to use the slow and sometimes inaccurate terraserver WMS service (which doesn't work with Potlatch anyway), there's another option:

There is at least one website with free online USGS topo maps with a Google Maps interface and tiles that are TMS compatible.

Just surf to this site, enable FireBug (on Firefox) or Web Inspector (on Safari) to analyze the resources and you'll be able to see the URL's for individual tiles.

Replace zoom/x/y.png by !/!/!.png in Potlatch as a custom background and, voila, fast and high quality USGS topo backgrounds.

keepright ( http://keepright.ipax.at ) is the single most important tool in the bag of open street map tricks, and pretty much the only reason I'm starting to believe that OSM may one day have a modest shot at being useful for routing (the lack of house numbers rules out a top-notch experience, IMHO). And now it's available for the US too. Woohoo!

The starting point of the keepright checking algorithm was in Philadelphia, so that's what popping up for first time keepright USA users.

If you ever needed proof that OSM is currently NOT ready for general routing use, look no further.

What A Mess...

US Interstates are often the first ways to be corrected (largest ! for $), but somehow Philadelphia didn't receive its share of TLC. It has interstates going straight through the city center, either on elevated structures over the local streets or in these huge ditches with local streets crossing above, but pretty much all of them have a junction as if they were intersection. You know: speed down on I-76 at 75 mph, next take a sharp turn right onto 21st St. Fun, fun, fun.

It's obviously no easy to have so many over- and underground highways, but you'd think that a city this size has at least some OSM fanatics to clean thing up?

Anyway, I've fixed up a large section of the downtown I-76 intersections, but some are too obscure to get a decent insight (like the one this post is linking to.) Let's get to work!

(And, please, whenever something is fixed, mark it as such on keepright.)

My question last week on how to use your own tiles in OSM didn't get an answer, so I had to figure out something myself. (http://www.openstreetmap.org/user/wallclimber21/diary/8019 )

This took a bit more effort than I expected:

Prologue: USGS topographical maps are a great source to map places that are otherwise difficult to map.
- They often contain features that are not available anywhere else (e.g. power lines)
- They are great for area's that are not visible on Yahoo Aerial tiles (e.g. too low resolution or because places where streets are covered by trees / forest).
- They often have feature names that can't be found anywhere else.
- They're in the public domain

(They're also very much outdated, so this won't work for areas with new construction etc.)

I've described here (http://www.openstreetmap.org/user/wallclimber21/diary/7750 ) how you can use those topo maps with Qgis.

Unfortunately, OSM support in Qgis is very basic. You really want to be able to use your tiles as a background in a real OSM tool.

The tiles generated by gdal2tiles.py follow the Tile Maps Service specification (TMS). But while they align perfectly with the tiles of Yahoo, Google, Virtual Earth etc, they don't follow the same standard in how to select a tile at a particular location. Google uses a zoom/y/x format, where (0,0) is at the top left of a flat world. Virtual Earth uses a recursive quadtree structure and TMS uses a structure similar to Google, but with (0,0) at the bottom left of the same flat world. Yahoo uses yet another slightly different convention...

The standard tiles of OSM are using the Google convention.

In addition, there are limitations to the way OSM tools can access custom background tiles: Potlatch only supports the Google convention (I believe), JOSM only supports the WMS standard, not the TMS. And Merkaartor supports both WMS and TMS, but with TMS only the Google convention.

I couldn't find an easy way to set up a WMS server, starting from existing TMS tiles. I'm sure it's possible with MapServer or TileCache or something, but I didn't have the patience to configure a whole setup.

So I decided to shoot for a Merkaartor based solution first.

Since this only supports the Google convention, one needs to hack gdal2tiles.py to modify the names of the directories and files that are created. You can find a patch for that here (http://gist.github.com/197749 ). Note that this patch only fixes the tile names. It doesn't make corresponding changes to the generated .html and .xml files, but since we don't need them, that's ok. A better patch would add a command line option to do all this. Feel free to contribute...

Now, go ahead an create some tiles from a topo map. You'll see that, after the patch, the y coordinate is different from before.

The next step is to get Merkaartor to use your tiles. Unfortunately, neither Potlatch, JOSM nor Merkaartor have support to use a local filesystem for this: they want to talk to a server to get the tiles.

No doubt, there are many HTTP servers that will just send you files, but I'm familiar with rails and didn't want to waste time, So I simply created a new rails project:
> rails topo_server
Then I created a symbolic link in ./topo_server/public directory to my tiles directory:
> cd topo_server/public && ln -s .../master_tiles master_tiles
Finally, I started the server itself:
> ./scripts/server

Finally, you need to tell Merkaartor to use your new tiles.
- Tools -> TMS Servers Editor (this dialog box is completely non-intuitive in how the buttons work...)
- Select one of the existing ones (e.g. Cycle Map) and click "Add"
- Modify the fields:
Name: USGS
Server address: localhost
Path: /master_tiles/%1/%2/%3.png
Tile size: 256
Minimum Zoom: 9
Maximum Zoom: 15
(Check you master_tiles directory for the minimum and maximum to fill in)
- click apply on the right size, then ok.

Now you can enable your tiles as a background by right clicking on MAP layer. A USGS option should show up under TMS adapter.

If all goes well, you're good to go!

Except, that is, if you're on a Mac...
Merkaartor will start to crash left and right and become completely unusable. This is because of a race condition in Qt (even the very last official release 4.5.2) with the way it handles Mac sockets. When you are using tiles hosted on your own machine (localhost), this bug will hit all the time and Merkaartor will crash hard.

The only way I've found to fix this is to download the Qt source code, apply the following patch (http://qt.gitorious.com/qt/qt/commit/84abdaa41e6c3bde6ac653e02bd72300b6681572?diffmode=sidebyside ), compile, wait a few hours and install. If you already have other Qt libraries installed, make sure you point to the new version by setting:
export DYLD_FRAMEWORK_PATH="/usr/local/Trolltech/Qt-4.5.2/lib::${DYLD_FRAMEWORK_PATH}"

(You don't want to know how long it took me to figure this all out...)

Unfortunately, my productivity in Merkaartor doesn't come close to Potlatch, which is nearly perfect in the way it is designed to minimize clicks and key strokes. Potlatch also has support for custom tiles. I assume it uses the same Google convention, but I haven't been able to get it to talk to my server on localhost. I'm still working on that. But, for now, the current solution will do.

Enjoy!

PS: It would be nice to have some info on the "New Diary Entry" page on how to add markup to a post. It's not rocket science to 'svn co' the full source code the OSM site and grep for the definition of 'htmlize', but others may not have the same kind of determination. (Edit: nope, didn't work either...)

I've created my own map tiles from USGS topo maps and created Google maps-like tiles for them with gdal2tiles.

Now I want to use them as an overlay in JOSM with the WMS plugin, but I can't get this to work.

I've looked at TileCache, but without success...

Has anyone tried this?

After adding a bunch of Yosemite trails to OSM, I've run out of my own GPX files to add more, yet there are so many more trails to add. Doing more hikes One way to get more is to go hike more, but that will only get me so far. Another is to get the trails from USGS Topo's (which are released in the public domain).

The USGS Topo data is often decades old, but for National Parks that's usually not an issue: nobody is their right mind is going to reroute the Mist trail, or the John Muir trail, or build a whole new trail from scratch in a National Park, so the topo maps are still very much up to date.

The accuracy of the topo's is usually spot on for POI's. Trails are not as good. A good example is a hike I did two weeks ago at Kings Canyon, where the trail on the USGS Topo is sometimes off quite a bit from a GPS recorded trail. At first, I thought this was due to the GPS unfriendly environment, but Yahoo imagery proved that the GPS was correct whereas the Topo trail was not.

That's said, there are good arguments to be made to use the topo trails:
* For generations, the topo's have been all that was available for those whohad to endure life without GPS.
* The accuracy is still good enough for hiking purposes. Sufficient to get guidance about the trail distance and where it goes and very accurate for trail intersections, which is really what counts.
* Google Maps: it turns out that the Yosemite hiking trails in Google maps are an *exact* copy from those of USGS topo's (* see below). If it's good enough for Google, it should be good enough for OSM until someone gets hold of real GPS data.
* The choice between no data at all or data that's entirely useable for practical purposes is a no brainer.

With that said, how to go about it?

JOSM and Merkaartor currently have some WMS support for USGS topo tiles. There are two problems however:
* The support is extremely flaky. That's a friendly way of saying: it just doesn't work most of the time. And if it does work, the performance, especially on Merkaartor is unbearable.
* The WMS tiles are derived from the terraserver-usa.com version of the USGS topos. They are generally ok, but, for some reason, they are a disaster in the Californian Sierra's. A quick look at Yosemite Village shows this: there are major jumps at original topo boundaries. The terraserver WMS tiles are unusable as an original source.

Here's a link with the Yosemite Village tiles:
http://terraserver-usa.com/ImageInfo.aspx?T=2&S=12&Z=11&X=339&Y=5226&W=2&qs=%7cyosemite%7cca%7c

A better solution: use the original topo tiles + QGIS

I've tried this only on a Mac, but I'm using nothing but open source tools, so it should be equally applicable for Windows or Linux.

I'm using the following ingredients:
* Original USGS topo files. Right now, I'm getting those from topoquest.com. Not the most user friendly web interface, but it allows on the download the original topo files in GeoTIFF format.
A typical filename will look like this: o37117g5.tif, which means: latitude 39, longitude -117, sub-quadrant G5. Each such file is typically around 12MB in size. You can download this particular file here: http://www.topoquest.com/map-detail.php?usgs_cell_id=50147
* GDAL: this is a library and toolset to work with raster based geographical data sets.
* QGIS: a program to interactively work with geographical data, create maps etc.

Installing QGIS is a bit of a pain (at least for Mac) in that it has tons of dependencies. You end up downloading and installing a whole bunch of libraries and frameworks before it will come to life. But with the instructions provided on the site, it didn't really give me problems doing so.

GDAL is one of those frameworks on which QGIS depends. However, I also had it installed from scratch (by downloading the source code and compiling) because some Python based script didn't seem to work out of the box.

The reason you need GDAL is to convert the USGS topo files to the correct coordinate system. The original topo files use a NAD27 reference ellipsoide and UTM coordinates. QGIS will read them in, but display them incorrectly with other, WGS84, data, so things won't line up. It may be due to my inexperience with QGIS that I couldn't get it to work, but eventually I took a different route: I converted the topo files from NAD27 to WGS84. This can be done with the following command:

gdalwarp -t_srs WGS84 o37119g5.tif o37119g5-w.tif

You can now read in those files into QGIS as a raster layer: Layer -> Add Raster Layer -> < select o37119g5-w.tif >

When you load multiple topo's at the same time, you'll notice that the white borders around each map will overlap with the one next to it. I'm currently writing a program to get rid of those borders, but for now, you can work around it by setting each topo layer as partially transparent. You will notice that adjacent maps line up exactly. MUCH better than the terraserver based WMS layers!

I wanted to make sure that existing GPX tracks of mine would line up correct. For this, you first need to enable the QGIS GPX plugin, as follows: Plugins -> Manage Plugins -> Enable GPS Tools in the list

You can now read it in the GPX file by doing: Plugins -> GPS -> GPS Tools -> Load GPX File

Your GPX data should now line up perfectly with the topo data.

What rests is to know how to digitize and how to create a GPX file out of QGIS.

For this, you first need to greate a GPX layer: Plugins -> GPS -> Create New GPX Layer.
After you've specified the file name (you have to add the .gpx suffix yourselve), you will see the gpx file appearing in the directory you specified.

By default, layers in QGIS are read-only. You need to explicitly open them for editing. Select the new gpx tracks layer and then do Layer -> Toggle editing.

You can now start tracing the trail that you're interested in. To complete editing a track, do a right click and fill in its name.

I couldn't find out how to save my changes and I had to resort to the QGIS manual to figure it out: switching the layer from edit mode back to read-only mode does the trick.

Note that saving the current project does NOT save the newly entered data. And quitting QGIS without leaving edit mode won't raise a warning that there's still unsaved data in a layer! DUH.

So that's it: you now have a GPX file with newly entered tracks.

All you need to do now is load them up in JOSM or Merkaartor to convert them to OSM format (and mark them as 'path', of course. ). You can't really upload the GPX to OSM as a trace, because the GPX file won't include time stamps. (Why this limitation?)

A fantastic improvement, maybe worthy of a Google Summer of Code project, would be to create an semi-automatic, user assisted digitizer: USGS topo's only use a limited amount of colors and black is reserved for names, roads and buildings. If a user could mark a initial point of a trail, it wouldn't be too hard to make a tool that digitizes up to the next intersection where it could then ask for further instructions. This would reduce the digitizing effort by an order of magnitude.

Have fun!

(*) To check that Google Maps data is an exact copy, one can do the following: GDAL has a really cool tool called 'gdal2tiles.py'. This takes a GeoTIFF file and converts it to a full WMS compliant tile set. It even includes a .html file that will create a browser page of the new tiles laid over a bunch of other tile (Google, OSM etc.) with selectable transparency. The Yosemite trails on Google Maps are an exact match.

I already filed a request for this with CloudMade about a month ago, but I just want to stress the importance of rendering ways that are marked as 'path' in full.

Right now, in the best case, tiles render the path but they don't render the path name. (OpenCycleMap renders the path name, but not the path itself, go figure.) In the worst case, unfortunately the default CloudMade tiles, paths are not rendered at all.

This makes the Cloudmade tiles very much unusable for all the mobile apps that are targetting hikers and bikers.

The goal of OSM is to create an all encompassing free maps database, but, let's be honest, when I'm in an urban environment, I'll use Google Maps, because that's what's all iPhone apps use and because it's information is complete enough. In my opinion, the real practical value for OSM is for those roads/trails/paths that are not covered by Google Maps. When I started with OSM in november last year, pretty much all open space preserve parks in the SF Bay Area were empty. I've added many hunderds of fire roads and single track trails everywhere and other have done the same. When you go there now to very popular parks for mountain bikers and hikers, a lot of them are now trail complete. It's exhilarating and something where Google Maps (and it's competitors) are absolutely nowhere.

For a mountain biker and a hiker, it's important to know if a trail is a single track or a fire road. We love the first and loath the latter. A single track is marked as 'path', a fire road is marked 'track'. I spend a lot of time annotating them correctly.

But in the last weeks, I've seen two cases where trails that I've added were changed from 'path' to 'track'. I sent a friendly email to the one who changed it. They were enthusiastic beginners at OSM who made the change because it would make a trail show up on CloudMade or OCM tiles. I can educate new users all I want about not tagging feature with the renderer in mind, but, think about it: it's pure coincidence that I stumbled into these modifications: they were in parks that were just a mile from my house. I don't know how many other incorrect modifications have been done because of this. All those changes are doing with the best intent: to make their features of interest show up on the map. Who can blame them?

(Maybe OSM needs some kind of change notification system, where interested parties get an email when something in a region of their interest has been changed?)

So, please, CloudMade, fix this. It would make a whole bunch of my iPhone map applications much more useful and will prevent people from making incorrect changes to the database.

Stepping off my soap box...

It looks like all US national parks are being converted from leisure=park to leisure=nature_preserve.

The wiki doesn't give clear instructions about whether or not that's not the official way to do it. In official US NPS parlance, a nature preserve is not quite the same as a national park, but I understand that's not really the point. However, whereas the park region used to be rendered in this nice, understated transparent light green (very similar to official NPS maps actually), the nature_preserve has this butt ugly high saturation green with, at higher zoom level, an equally ugly 'NR' grid rendered onto it.

I know, I know, don't tag with the renderer in mind, but won't someone please think about esthetic sensibilities and all that?

When I looked at OSM for trip planning in Namibia, I simply assumed that the empty spaces on the slippy map meant that the data just wasn't there.

Now, as I try to upload some data, it turns out that it's actually very complete: pretty much all major roads in the country are there!

It's just that they are marked as "highway:tertiary" and "surface:unpaved". Except for a handful of main arteries, all roads in that country are, indeed, unpaved gravel roads, but since you can often drive over 100km/h on them (that's actually the official speed limit, even if it's definitely not always a smart thing to do) and since they're the only way in which cities (by Namibian standards) are interconnected, it's hardly fair to call them tertiary roads...

A nice example is here: http://openstreetmap.com/?lat=-24.426&lon=16.1727&zoom=12&layers=B000FTF

No matter now deep you zoom in, you'll never be able the discover that you're actually looking at the intersection of roads D854 and D850... The labels are never rendered.

In all honesty, those roads don't even show up on Google Maps, but at least it shows the C roads (highest standard gravel roads) in all its glory and some cities.

Also, so other D-roads are tagged as secondary and will show up if you zoom in very closely.

It's probably pretty hard to implement, but a possible solution would be to render detail density adaptive: when there's little or no roads in the area, label roads and mark cities even if they normally wouldn't show up in densly populated area.

As similar example is here: http://openstreetmap.com/?lat=-23.8931&lon=16.0022&zoom=13&layers=B000FTF

Solitaire is a hamlet (as tagged on the map), probably of no more than 100 inhabitants, if that much. But when it's the only place with a shop and gas in an area of, say, a hundred kilometers, it probably makes sense to make it a very prominent hamlet indeed, as is the case on Google Maps:

http://maps.google.com/?ie=UTF8&ll=-24.557116,15.869751&spn=2.792765,4.839478&z=8

(Yahoo Maps is far worse then OSM, BTW!)

In a previous version of Potlatch, setting CapsLock to ON resulted in roads etc to be come semi-transparant, which made it really easy to check if OSM data correlated with Yahoo Map data.

That doesn't seem to work anymore?

Henry Coe, 40 miles south of San Jose, is *huge* and there's a million small and not so small trails going everywhere. I had already spent quite a bit of time adding trails the last few weeks but today, I spent my sick day tracing my whole inventory of Coe GPX files. This park used to be pretty much empty when I discovered OSM about a month ago. Now you don't know where to look first.

Unfortunately, I don't have the names of many trails. Will need to take better notes the next time...

*dupe*

Added a whole bunch of trails to Alamden Quicksilver park. Purisima Creek Open Preserve is now almost completed (except for some hiking-only trails). I've also added a trail on South Mountain, Phoenix and aligned a bunch of TIGER data in the surrounding streets.

Has anyone thought about importing the GIS data from the US National Park Service? It's weird that the outline of parks such as even Yosemite are approximations at best, but often completely non-existent (e.g. Golden Gate Park). I'm not 100% sure, but since this is data provided by the federal government, this should be free to use.

Speaking of which: pretty much all map providers (e.g. AAA) will display National Parks prominently on their maps, but Mapnik only does so at fairly detailed zoom levels. Am I the only one who uses National Parks are a way to orient myself on a map?

Today, we mountain biked all trails of St. Joseph Hill's Open Space Preserve in Los Gatos. I've added them to the map. One more local park that's been fully mapped for hiking and biking.
I also added a couple of trails in the nearby Almaden Quicksilver park (and deleted some ridiculous TIGER segments in the process.)

Spent the whole evening correcting streets in Cupertino and Saratoga. Sometime the TIGER data is 100% accurate and then suddenly the streets right next to it are seriously off...

I just correct points based on the Yahoo photos and don't touch any of the TIGER tags. I hope it's not necessary to touch the 'reviewed' flag to avoid changes to be undone with a later import? There's an edit history after all, right?