Press "Enter" to skip to content

Category: mapping

JOSM Tip: Simplify Way before Improve Way Accuracy

Consider the following: You are attempting to update OpenStreetMap (OSM) trail routes using JOSM and find that the previous way is very detailed, but fairly wrong, meaning that a lot of nodes will need to be moved.

Even with the Improve Way Accuracy tool this’ll be a pain. So what can you do? First decrease the number of nodes using Simplify Way and then move the remaining nodes, adding new ones as needed in the gaps. This will keep the original way intact, keep most of the route present, but allow for easy cleanup. It also reduces the number of nodes, making for simpler routes that take up less space on GPS devices. (I find that a maximum error setting of 0.5m or 1m works well.)

In 2016 I used the official GIS data from the Noquemanon Trails Network (NTN) to add the singletrack trails to OSM. This worked pretty well, but since then it’s become possible to trace the Strava Global Heatmap high-resolution data when mapping. When doing some routine updates and using this layer for assistance I noticed how many trails originally entered using the NTN’s official data aren’t quite correct. So along with adding changes, I’m tweaking the trail routes using the Strava data.

The primary issue is that the official data would often have a large number of points very close together — in some cases just inches apart — particularly around curves. These points were much closer than needed for accurate mapping, and yet these curves would be the main things that needed adjusting. Moving all of these points would be a hassle and the resolution wasn’t necessary, so by simplifying the route, correcting the nodes that remain, and adding in more as needed, cleanup of the route is much faster. It also reduces the number of nodes along each way, saving space.

The following images show a great example of this problem along Mossy (way 40781586), the last piece of single track in Pioneer Loop (relation 6109593) when ridden clockwise from the trailhead:

Detail of original data for Mossy in JOSM. Note the very detailed, yet inaccurate, curves.
Mossy after simplifying the way with 1m maximum error.
After manual cleanup of the simplified Mossy using the Improve Way Accuracy tool
Comments closed

CRAMBA Trails Outline Poster from OSM Data

Finding myself a little bored, I put together a poster (11″ x 17″) showing outlines of the CRAMBA-supported trails on one overview. (Link)

This ended up being more popular than I expected, with a handful of people wanting to know how I did it, so I’ll detail the steps here:

  1. Ensure that all the trail routes are in OpenStreetMap.
  2. Using JOSM load each trail area one at a time and make an OSM XML file with just the data you want outlined:
    1. Select the ways which comprise the trail you want shown.
    2. Create a new data layer (Command-N).
    3. Make the original data layer active.
    4. Copy the selected data from the first layer to your new layer with EditMerge Selection (Shift-Command-M).
    5. Hide the original data layer.
    6. Review the new layer to be sure it has everything you want.
    7. Select all nodes and ways (Command-A) and remove all tags to make later processing easier.
    8. Look good? Is everything you want in the new layer? Save it to a .osm file and do the next trail.
  3. Once you have an OSM file for each trail, convert them to Adobe Illustrator format using this version of osm2ai.pl.
    1. Get osm2ai.pl working on your computer. I run this on macOS, and it works fine on Linux as well. Since it’s a Perl script there are probably some dependencies; likely resolved by installing a few modules.
    2. Process each OSM file with: osm2ai.pl --input infilename.osm --projection mercator --output outfilename.ai
  4. Open each file in Illustrator, combine them into a larger document, make it look the way you want, etc.
  5. Done!

Comments closed

High Resolution Strava Global Heatmap in JOSM

Mapping trails in OpenStreetMap (OSM), using JOSM, is done by overlaying one or more data sources, then hand-drawing the trail locations using this background data as a guide. While not a default in JOSM, with a little know-how and a paid Strava Subscription the Strava Global Heatmap data for this can be used as well.

While there’s a fair bit of info about doing this scattered across some JOSM tickets (eg: #16100), this post is to document how I make this work for me by creating a custom Imagery provider. Because there isn’t an official (and robust) plugin to authenticate JOSM against Strava it’s a little tricky, but overall isn’t bad.

First, you need to sign into Strava, then go to the Global Heatmap and get the values of the three cookies CloudFront-Key-Pair-ID, CloudFront-Policy, and CloudFront-Signature. These values are essentially saved credentials, and by using them in a custom Imagery provider URL, JOSM will use these to access the Global Heatmap data via your account and make it available as an Imagery layer.

The easiest way to get these cookie values is to use the Developer Tools built into your web browser:

  1. Launch your web browser, either Firefox, Chrome, or Edge.
  2. Go to the Strava Global Heatmap, sign in if needed (check the Remember me checkbox), and be sure it’s properly displaying high resolution data.
  3. Load the developer tools for your browser to show cookie details:
    • Firefox: Click the Hamburger MenuWeb DeveloperStorage Inspector (or press Shift-F9), expand Cookies and click https://www.strava.com.
    • Chrome: Click the Three Dots Menu → More tools → Developer tools (or press Ctrl-Shift-I), select the Application tab, expand Cookies, then click https://www.strava.com.
    • Edge: Click the Three Dots Menu → More tools → Developer Tools (or press F12), select the Storage tab, expand Cookies, then click heatmap.
  4. Get the value of three cookies, CloudFront-Key-Pair-ID, CloudFront-Policy, and CloudFront-Signature, and make note of them (in Notepad or such). You’ll need them to build the custom URL. Here’s examples of what the three cookies will look like:
    • CloudFront-Key-Pair-ID: VB4DYFIQ7PHQZMAGVMOI
    • CloudFront-Policy: FTUGYikErADOqGmqEnPTaaZ3LCE2QbOJSSsmPD9vdY0hfjSWWHZBgnRai1aQcvUe
      2YtgkG3KdP8IencRaGrS3I5bHAeabQ4I776T7UipDPFzNVh9PBIRvb3DjDqVfssp
      mBA5IrxacQGfB2YmKwegp9OTBfUFkvtEQOlzeXfqeuc34cp408GYXSvp2DsnxKFh
      ofbwygraae3VZqnZxp3vBpz3lBoyPnBEN1djJkxpU1fF_
    • CloudFront-Signature: lrMJG0L6zJilpSdGEnWug0zskcMDD5VVGdSRu~P4OOVp5z8iCMBRBO040wxunEQQ
      p0wVHdaCaTFAVqiM0jC0AApW7HigXjx57nsoEVslOzRuoX9S3g-
      FCePWYNPyZcJ95~6SoGdNRHz-
      b-ZZxk4ERhjzXCNIEE8Gt5uImitcPN3HRlhXexS39g30~5OEv~4FHzXfOOIhQ5X5
      cGUD0SpdYviBE~vkAJCk3TbQMXxlJbyX4OOL5xvVPv5WjzMKTuvEk6Z9m6iFBbZc
      8ov4qS60th0tZ1RM4pNMoIWKoqPS3~UPGISe6Xa8Kq-
      tQPxHlfGciI5uhoTp4b7lPussh52QaN__

(Note: These are not real values, you’ll need to look up your own.)

Next is to add a custom imagery provider for the Strava Global Heatmap to JOSM using a URL that you build with these cookie values:

  1. EditPreferences… (F9)Imagery preferences
  2. Create a new TMS string by replacing the [] sections in the following string with the cookie values from above, without the []s:
    • tms[15]:https://heatmap-external-{switch:a,b,c}.strava.com/tiles-auth/all/hot/{zoom}/{x}/{y}.png?Key-Pair-Id=[CloudFront-Key-Pair-ID]&Policy=[CloudFront-Policy]&Signature=[CloudFront-Signature]
  3. Add a new TMS entry by clicking the +TMS button.
  4. Paste the string from Step 2 into Box 4, Edit generated TMS URL (optional), and then give it a name in Box 5, such as Strava Global Heatmap High-Res.
  5. Click OK twice to return to the main JOSM window, and then try to use this layer over some existing data, just as you would aerial imagery.

If it doesn’t work, double-check your URL and be sure the entered cookie values are right, including the underscores after CloudFront-Policy and CloudFront-Signature. Also be sure you haven’t logged out of Strava, as this will expire the cookies. (If you make changes to the URL in JOSM you will need to delete the existing imagery layer and then re-add it to have the new URL used.)

These cookie values, in particular CloudFront-Signature, will occasionally change as cookies are expired and reset or if you log out of Strava. If things were working and then stopped, you may need to get new cookie values from your browser and update the TMS strings.

By default the TMS URL we started with shows the default heatmap, for all activity types, with Hot coloring. Depending on what other data you are working with it may be useful to show just Ride or Run data, perhaps in different colors. In the TMS URL, the first part after tiles-auth is the type of data, and the second is the color. By using this format, replacing [data] and [color], you can create additional heatmap layers:

tms[15]:https://heatmap-external-{switch:a,b,c}.strava.com/tiles-auth/[data]/[color]/{zoom}/{x}/{y}.png?Key-Pair-Id=[CloudFront-Key-Pair-ID]&Policy=[CloudFront-Policy]&Signature=[CloudFront-Signature]

Valid values for type of data: all, ride, run, water, winter.

Valid values for heatmap color: hot, blue, purple, gray, bluered.

Creating multiple layers, riding-only and running-only, of different colors was extremely useful when mapping Avalanche Preserve Recreation Area in Boyne City as cyclists tend to stick to the mountain bike trails and runners to the hiking trails. While I had orthorectified maps from both the city and TOMMBA, the separate riding and running heatmaps made the routes much clearer. In the example image above (link) I have the Ride data as color Hot and the Run data as color Blue; perfect for illustrating the mountain bike vs. hiking trails.

If you’d like to see this area as an example load the are around changeset 85690641 or a bounding box with the following values and see for yourself:

  • Max Lat: 45.2077126
  • Min Lat: 45.1878747
  • Min Lon: -85.0339222
  • Max Lon: -84.9832392
Comments closed

Using OsmAnd Offline Maps during Remote Rides

Many of the popular, long cycling events — or even just solo rides — require a GPS so that one can follow a route in order to navigate. While most of us have bike computers that will show the route, most folks don’t have a GPS with the kind of detailed base maps that become useful for detours, emergencies, or just as an additional tool… Much less something that works when outside of cellular coverage areas.

For this, I recommend you look at offline maps using OsmAnd.

I have (and love) the Garmin Edge 130 as it’s perfect for my riding and training. It’s been great for displaying recording data and following routes from the unmarked 100 mile versions of The Crusher and Barry-Roubaix, to augmenting signage in Marji Gesick and following a route through remote parts of Batchawana River Provincial Park. But occasionally I need a bit more; something to help find a detour around a flooded road, a quick way back to the car in a rainstorm, or just a sanity-check of which branch to take at a fork on the road.

Sure, I could buy a super-high-end bike computer that features detailed base maps, but I rarely need these and don’t want the physically large computer on my bike. So I use my phone, running OsmAnd, loaded with offline maps that display without requiring a data connection. Coupled with putting one’s phone in Airplane mode (turns off the cellular and WiFi radios and saves battery — yet GPS still works) a modern phone will typically get days of battery life while sleeping, with maps and a route quick to access.

Whenever I actually need to see this map I simply pull it out of my pocket or bag, wake it up, get what I need from the map, and carry on. No screwing around with a small touch screen and funky zooming on a Garmin or Wahoo. (I believe that, to date, neither Garmin nor Wahoo have made map perusal as straightforward as it is on a basic mobile phone.)

This is also a great way to get maps on your phone when traveling somewhere costly for cellular data, like during a day-trip to Canada.

Whether you’re doing Marji Gesick or The Crusher or just following a route found on a cycling club’s website, I recommend augmenting your navigation with offline maps in OsmAnd by doing the following:

  1. Visit the OsmAnd site and follow the links to your phone’s store to install the software.
  2. Get a copy of the GPX file on your phone; for Android phones this is often as simple as visiting the download website and downloading the file.
  3. In OsmAnd‘s menu, pick Download Maps and download maps for the regions you want. I personally have Standard Maps, Contour Lines, and Hillshades for Michigan installed all the time, and then if traveling will install for other states/provinces/countries.
  4. Once the offline maps are installed, pick Configure map from the menu, then Map source, and ensure Offline vector maps are selected. The default, OsmAnd (online tiles), is an online map source that requires data.
  5. To display a particular route, pick GPX files…, Add More…, and then browse your phone to add the downloaded GPX file. Ensure this file is enabled (checked), and then go back to the map screen.
  6. You’ll now see the route overlaid on offline maps, easy to zoom in and out of and find your way around. Try turning on Airplane Mode and see that it all still works.

Once I have everything set up, here’s some tips about how I actually use OsmAnd while out on my bike:

  • If in an area with minimal or no cellular service, put the phone in airplane mode so the battery doesn’t get used up while it searches for a signal.
  • Turn off all lock features on my phone (fingerprint / pattern / PIN unlock), so I can wake it up with just a press of the power button.
  • You can use this at the same time as Strava, and Strava generally works in Airplane mode as well.
  • If I’m at a spot where I’m not quite sure which way to turn (based on the single line on my Edge 130), use the map to figure out which branch to turn on. Even the remotest of fire roads and two track are often mapped.
  • If I think I might have missed a route sign (say, in Marji Gesick) I’ll use OsmAnd to see if I’m still on route, and to get back if needed.
  • Look for other nearby roads or trails if I need to detour or shortcut back. I had to do this yesterday during the Founders Fall Fondo as part of the 62 mile Barry-Roubaix route was flooded.
  • Many small, old, sometimes abandoned, yet somewhat navigable roads are still listed in the OpenStreetMap (OSM) data that OsmAnd uses. While these aren’t great for cars and won’t show up on things like Google or Apple Maps, they are often quite useful when on a bike. Look to these for exploring / finding new routes.
  • Many trails open to mountain bikes are included in the OSM data. This includes all CRAMBA trails and most trails in the NTN and RAMBA areas. (Note: Trailforks is a better source for discovering MTB trails themselves.)
  • Periodically update maps to get new OSM data. As roads and trails change, maps get updated.
  • Pay For OsmAnd+ Live ($5.99/year) to get access to more-frequent, automatic map updates and support the open mapping community. 

While this is just a basic overview of using OsmAnd and how I use it when cycling; it does a whole lot more including recording routes, navigation, editing map data, sharing location between contacts, and more. Give the online Help and Features a look for more information.

 

Comments closed

Trailforks, Pinkbike, and OSM

For years I’ve been pretty enthusiastic about OpenStreetMap (OSM) and using it to map trails (MTB and otherwise). While there are a bunch of other ways to map trails online (Google Maps, MTB Project, Trailforks) I have stayed away from contributing to them because of the one-way nature of submissions; your contributed data gets locked behind their license. While MTB Project and Trailforks both claim to allow some manner of reuse of data, it’s nothing as useful as OSM‘s Creative Commons (CC) based licensing. Effectively being the Wikipedia of GIS makes it extremely useful for those of us who want to both contribute data and build open maps on the larger set.

Then suddenly last night I read this article on Pinkbike discussing how they took OSM data, parsed it to highlight mountain biking routes, and are now using it as the base map for their Trailforks mapping site. They built a tool on top of the open data and made something great.

This is really, truly excellent.

This sort of reuse of public, open data in OSM is the exact reason why I contribute to it. The folks at Pinkbike / Trailforks have taken a useful set of data from all over the world, processed it, and made something good. This would not have been possible with the data locked up in Google, MTB Project, or even the stuff contributed directly to Trailforks.

I look forward to where this’ll go. The Pinkbike article mentions that they’ll be reimporting the data a little different in the future, and talks about how they are going to have another article about tagging to better support Trailforks. While OSM has some minimal standards for MTB tagging (eg: mtb:scale:imba) I look forward to a bit more de-facto standard around this.

Leave a Comment

New Trail Map of Bald Mountain Recreation Area – North Unit

This morning I’d originally planned on going for a ride in the currently-active snowstorm, but I started working on a project that I’d wanted to complete for a while, and this evening I completed it: a CRAMBA-IMBA map of Bald Mountain Recreation Area – North Unit. Home to some gently rolling, old-style trails built with cross country skiing in mind it’s not very challenging, but this is one of my favorite mountain bike trails in the area.

Over the last few slow speed wintertime rides there I’d collected the GPS data I was lacking, put it into OpenStreetMap today, cleaned up the existing data, and followed my mapping workflow to create this map. It has a fair bit more detail than the official park map, showing some of the unofficial but well used connector trails, colloquial names for some areas, and a bit more context. It’s what I personally would want for a mountain bike centric map of the trail.

So, what to map next… I’d like to do a CRAMBA-IMBA map of Pontiac Lake Recreation Area, but I’ll need to spend more time riding and exploring there before that’s feasible. I’m familiar with the trails, but not familiar enough with the park to do a proper map. I also want to update the Bloomer Park map to get something that better matches the style of the other maps. But, for now, I’m glad that this got done. I feel like I’ve accomplished something.

One can see this map overlaid on Google Maps, along everything else in the MMBA Trail Guide here: link.

Leave a Comment

Crooked Maps?

In working on the forthcoming MMBA trail guide data I’ve had to find the lat/lon boundaries for rectangular maps so they can be presented as overlays on a Google Map. In doing this I’ve found that almost all of the Michigan DNR-drawn maps seem slightly crooked compared to what I see in OpenStreetMap. It almost looks as if the designer of the map rotated it slightly so that the north/south roads align with the page border.

When making the Stony Creek and Addison Oaks maps I’d noticed that roads which I’d previously thought of as oriented perfectly north-south or east-west were slightly crooked. Now I’m wondering if there’s something I’m not understanding with regards to map projections in Michigan.

Reading this article from the DNR about map projections I see that Michigan has its own projection system called Michigan State Plane Coordinate System. (More info here at Wikipedia.) So, I’m starting to wonder if there’s something that I’m missing and possibly doing wrong with regards to map making in Michigan. I just don’t know enough to know yet.

Leave a Comment

osm2ai.pl w/ JOSM XML Support

When working up some map stuff and trying to follow my own OSM workflow I ran into a quirk where the osm2ai.pl script wouldn’t deal with OSM XML coming out of JOSM.

Thank to help from a few folks on IRC I now have this updated version of osm2ai.pl available and it will successfully parse files from JOSM.

The root cause of this problem is the script parsing XML by hand, but as I didn’t have the time (or knowledge) to fix the problem by using an XML parser, this hack which deals with differing quote types and extra spacing, was put into place. It works, and I was able to get the map that I promised to someone complete.

Leave a Comment

Muddy Trails Enhance OpenStreetMap Data

Part of my map making workflow that uses OpenStreetMap (OSM) data requires updating the existing data set to be as accurately as possible before exporting the data. The data that I originally loaded into OpenStreetMap for River Bends was based on GPS surveying that I did not long after some new trails were built. Thus, the (prone to error) GPS data was the most accurate data available.

When updating OSM data, one is explicitly permitted to trace Bing imagery to enhance maps. Earlier today while poking around in JOSM (which adds Bing data as a tile so it can be traced) I noticed that the latest imagery for River Bends was taken on March 11, 2012 and most of the recently built single track is now visible. Because of the particularly wet spring here in Southeast Michigan many of the trails were muddy while these photos were taken, resulting in the trails being visible dark marks on the traceable photos.

The image above demonstrates this, showing the imagery date, current OSM data (red dashed line), and the wide/dark lines are the trails themselves. By adjusting the routes to match the imagery I can radically clarify the OSM data, validating and refining routes. River Bends is due for a map update soon after some new trail construction is complete, so this means that the next map of there will be much, much more detailed and accurate. Thanks in large part to a wet, muddy March.

Leave a Comment

Addison Oaks MTB Route in OpenStreetMap

With the upload of OpenStreetMap changeset 12787189 I have finished adding all of the hiking and equestrian trails, intersection markers, and mile markers needed to draw a second generation version of CRAMBA’s Addison Oaks map. This data includes the still-under-construction connector trail which’ll create a new park entrance at Indian Lake and Lake George and provide a safe route to Addison Oaks East and Oakland Township’s Cranberry Lake Park. I also added all of the named routes through the park as found on the official park map, including the specific hiking, biking, and equestrian permissions and restrictions.

If all goes according to plan, the next version of the map will show the actual locations of the C and D trails, shared hike/bike/equestrian (on C only) areas in the west of the park while highlighting the MTB route. The mountain bike route (trail F) frequently crosses these trails and occasionally shares their route, so I want to better illustrate the interplay between them all. This next version will also be based on OpenStreetMap data, which beyond the benefits of being CC BY-SA licensed should make it easier to update and use for future planning.

The only things I have left to add to OpenStreetMap before it’ll be usable for drawing a complete map of the park are a few campground roads and the B loop, a relatively flat and straight hike/bike loop around the campground in the northeast of the park. I’ll probably record that route the next time I’m at the park.

2 Comments