83 search results for "online"

Making an Online RAMBA Trails Map

Published September 22, 2021

RAMBA Trails Map v1.0 (Default View w/ Epic Loop Layer)

Introduction

Kristen and I have been spending a good deal of time in the Ishpeming and Negaunee area this year, and I’ve made it a personal goal to become more familiar the local trails — both RAMBA-supported and otherwise — and get them documented OpenStreetMap (OSM). Having these trails in OSM provides two big benefits: they appear in other mapping tools (such as OsmAnd, GaiaGPS, Strava, MapMyRide/MapMyRun) and the trail data can be freely used to build other tools.

Over the years of making trail maps with OpenStreetMap I’ve mostly produced PDFs for printing, leaving mobile and online mapping to other apps. These work well, but have the big downside of rendering routes with their style. That is, online maps via these tools’ show the routes, but look quite different from print maps, even if all the data for them to display more data (such as colour=* tags on relations) is in OSM.

While these apps work pretty well, and I use them myself routinely for navigation, I got the itch to see if I could make a web-based map that looked more like locally produced print maps than app-based renderings. It seemed like a good project, a good way to learn some basics of modern web development, and maybe make something useful.

What I ended up with was a slippy map of showing the RAMBA trails that uses layers of pre-rendered tiles to show the different official trail routes, placed over a background map. The map viewer is client-side JavaScript that loads static tiles from a basic web server, making this a very simple app to host (just a bunch of static files on a site).

In this post I intend to document the major steps of how I made this map, why I used the tools I did, and share the code to reproduce (and update) this build. Hopefully this’ll allow others to get their head around these map presentation basics, perhaps even reusing this work to make and host another map.

Update OSM Data

Mostly outside the scope of this article but worth a mention, a significant amount of time was spent ensuring that the RAMBA area trails are accurately listed in OSM. Without good data it would not be possible to go further, as the OSM data is the base data used to create other maps.

By combining information from a bunch of sources, and doing some personal surveying of trails while riding and hiking, I was able to get all of the official RAMBA trails documented, along with numerous other paths and tracks in the area. This building a complete picture of the usable trails in the area.

Information used to get the RAMBA trails in OSM included:

My own recorded ride/hiking data, notes, etc.
Strava Heatmap data.
Official route files (GPXs) from events such as Marji Gesick and Polar Roll.
The official RAMBA map.
Hand-annotated map from Danny Hill listing local trail names.

These sources were combined in JOSM, cross-referenced, ways drawn and tagged, relations built out, and before long a complete picture of the RAMBA-area trails — official and otherwise — were in OpenStreetMap.

Most importantly, beyond documenting the trail locations, trails were grouped into route relations to show each official route, and then all the official routes were grouped into a superroute for all the RAMBA trails. As of time of writing, relation RAMBA Trails (12425503) is the superroute that aggregates the individual trail routes such as Epic Loop (8467869) and Malton Loop (8468010).

The result of this is accurate trail data that’s easy to query for and style using other tools.

Rendering Tiles with Maperitive

There are myriad ways to render tiles from OSM data, with most of these involving setting up a database server and a toolchain which’ll generate, cache, and serve tiles on demand. For most large data sets this makes a lot of sense, but for a small trail system I really wanted to use static tiles I could serve from a simple webserver.

Eventually I came across Maperitive, a desktop application for Windows that takes GIS data (including OSM), stylizes it with a relatively simple ruleset, and can generate tiles in the standard XYZ format for use elsewhere. It can also be scripted, which meant I could use it as part of an automated workflow to generate new tiles as the OSM data changes. This seemed like a good solution, so I set about writing some rulesets that would reasonably show the RAMBA trail routes and some automation around it all.

After a lot of experimenting I settled on a having separate tile set for each of the official loops, an overview of all trails, and a base map. The base map would always be shown, and a user can toggle between layers which highlight all the trails or individual loops.

After a few iterations of custom rules, I settled on a simplified set based on the Default.mrules file which comes with Maperitive for rendering the base map. The only modification was changing the font to Michael Adams’ Roadgeek 2005 Transport Medium font, as it looks nicer than the default, Verdana. For the overview and route layers I created simple rules based on the the default rendering of highway=path, using the Heavy version of the font. The rule for each trail route (relation) selects the trails in a given relation then colors them accordingly.

Creating these rules took a bit of fiddling, as Maperitive is both a bit of a dead project, not completely documented, and (in the latest Beta) sort-of buggy where sometimes the map display would stop updating. Still, even though I’m not great at making attractive things, I was able to come up with something that worked well enough.

Conveniently, Maperitive also comes with a command line version (Maperitive.Console.exe). After settling on rendering rules and a tile generation script, I used this as part of an automated workflow which downloaded OSM data directly then rendered each of the tile sets.

After tile generation I used a Windows binary of OptiPNG to losslessly compress the tiles, resulting in a ~62% space savings (original: 746MB, optimized: 286MB) which’ll reduce storage and bandwidth overhead.

The Front End

With tiles generated I needed a way to display them. It turns out that OpenLayers was easy to use and it all ran as simple client side application in a browser. By using npm and parcel, with Visual Studio Code for editing, it was quite easy to get the site developed, tested, and bundled up for deployment. The only component I had to add was ol-layerswitcher control, which provides an easy way to toggle between layers.

Prior to this I had very little experience with modern web development, with my exposure to JavaScript pretty much limited to reading others’ code to figure out what it’s doing. After a bit of confusion (and having to accept the hidden complexity of using an application bundler), I was able to focus solely on writing a single main.js file with a basic index.html that together do what I wanted:

Run full screen by default.
Show all trails by default, with toggles for the defined routes (layers of the map).
Show an attractive background map below the routes to show the rest of the area.
Offer controls to use geolocation to showing one’s location on the map and reset the view to the original map extents.
Look sane on desktop and mobile devices.

This ended up being much easier than I thought, and between the OpenLayers Examples and just some basic programming I was able to get something I’m happy with. Far more time was spent designing the tiles and thinking about what I wanted it to do than writing the code to display it all.

Tile Hosting

The actual map tiles are a number of small PNG files, and a typical session of viewing and panning around the map can result in hundreds of image loads. This was seeming a bit slow when being served from nuxx.net via HTTP/1.1, so I looked into using HTTP/2 to improve performance.

Unfortunately, it was not simple to turn on HTTP/2 here at nuxx.net as I’m using PHP for WordPress, which in turn requires MPM prefork, which precludes mod_http2. I could have set up another web server and such, but for now I’m hosting the tiles in AWS, with the tiles uploaded to an S3 bucket and served via CloudFront.

This should allow for better tile download performance than what I can do from my server. Despite potentially incurring a bit of a financial cost it is a good experiment in hosting tiles in the cloud. I may change this in the future, particularly if it becomes cost prohibitive, but for now it’s working well.

Follow Along At Home

If you would like to generate this same map, start by downloading this ZIP file: ramba_trails_map_code_1.0.zip. It contains the scripts and rules needed to generate the map tiles (ramba.mscript and the .mrules files), the index.html, main.js, and package.json for the OpenLayers-based front end, the .osm file used to generate the first release of the map, and a few batch files that tie it all together.

You will need to download Maperitive (latest beta: link — from this post / mirror), curl, OptiPNG, and the Roadgeek 2005 fonts to generate and optimize the tiles. These scripts may work fine with older/release versions of Maperitive, but betas incorporate a bit of collision detection for text, making things look nicer.

These batch files are included to will help you out, but may need some editing to fit on your environment:

fetch_osm.bat: Uses curl to download all OSM data within a bounding box that encompasses the Ishpeming/Negaunee area.
generate_tiles.bat: Runs ramba.mscript via Maperitive.Console.exe to generate the tiles.
optimize_tiles.bat: Copies the unoptimized tiles from the .../tile_output/raw output directory to the .../tile_output/optimized directory, then runs OptiPNG against the tiles to optimize them in place.

To build the web app you’ll need to install npm, parcel, create a new OpenLayers app as per the directions here. Then install ol-layerswitcher (npm install ol-layerswitcher), replace the default index.html, main.js, and package.json with the ones I provided, and you should be ready to go.

Updating the Map

As you can see, the map is two major pieces: the front end and the tiles. Whenever the map data changes in OSM the tiles can be regenerated to update those layers. The code for the front end web app only needs to change if the storage location changes, features are going to be added, etc.

Conclusion

This map has worked out rather well and I’m happy calling it v1.0. It’s been a great learning experience, and I’ve even managed to produce something useful that didn’t exist before: an interactive map of some of the most rugged single track trails in Michigan; one of my favorite places to ride mountain bikes.

It’s far from perfect, and there are some things I could do differently, but for now, I’m considering it a success. When in Negaunee for vacation last week I successfully used development versions of this map to find my way around, so I know it’s better than nothing.

If you find any quirks in the map data — such as trails with wrong names or in the wrong location — please take a screenshot and show me what’s wrong and email that to steve@nuxx.net. I’ve done my best to ensure the RAMBA trails are accurately mapped, but I’ve certainly missed some things.

Problems

No key or other ancillary information (such as logos) as are normally found on print maps.
No terrain. While 1m DEM elevation data is available from the USGS, I couldn’t figure out how to use it in Maperitive for generating hillshading.
No easy way to add clickable items to show additional info, link to external map apps (eg: for navigation).
Maperitive’s text rendering isn’t the best, resulting in goofy looking text at some zoom levels.
Long trails only have one label placed on the middle. Trails with one name broken into multiple ways will be labeled numerous times.
Due to being run in a browser it’s a sufficient, but not great, mobile experience. Specifically, selecting the geolocation, recenter, and layer controls can be fiddly because they are so small.
Does not work offline, but thankfully most of the RAMBA area now has good mobile data coverage.

Things To Investigate

Keep an eye on AWS cost and performance.
Look at Leaflet for the front end, as it seems a bit more modern.
Consider rendering map tiles with TileMill. This will add a lot of complexity both in setup and styling tiles, but once done should allow a lot more flexibility in styling and overcome most of Maperitive’s problems. mapbox/mbutil should work for getting XYZ PNGs out of MBTiles files.
Consider using a tile server if I don’t want to deal with discrete files.
Look more into using vector tiles with client-side styling. (I passed on this for now, as a GeoJSON file showing each of the route is a large download and had no benefit over raster tiles.)
Maperitive should run under Mono, and OptiPNG is available for many platforms, meaning it should be possible to reproduce this build under macOS or Linux. Note that the GUI for Maperitive will not currently run on macOS due to Windows.Forms currently being based on Carbon, which is not available for 64-bit macOS. So while the CLI should work, the GUI version isn’t currently compatible with macOS 11.5 (Big Sur) and higher.

Comments closed

BorgBackup Repository on DSM 7.0

Published November 28, 2022

A few years back I began using Borg for backing up nuxx.net, sending it home to my Synology DSM 1019+. At the time this was running the 6.x family of DSM and worked great, but it broke after moving to v7.0. Attempts to run Borg would result in this error:

/var/services/homes/borguser/borg: error while loading shared libraries: libz.so.1: failed to map segment from shared object

This appears to be happening because with the upgrade to v7.0 /tmp is mounted noexec.

adminuser@diskstation:/var/services/homes/borguser$ mount | grep /tmp
tmpfs on /tmp type tmpfs (rw,nosuid,nodev,noexec)
adminuser@diskstation:/var/services/homes/borguser$

While a few online solutions (such as this one) propose remounting /tmp with exec, this is a poor solution as it changes the security model for DSM v7.0 and may break in the future during an upgrade. The best solution for this is to create a private temp directory for just borguser and define it as $TMPDIR.

To do this create ~borguser/tmp, ensure it’s owned by your Borg user, and set it to 700:

mkdir ~borguser/tmp
chown borguser:users ~borguser/tmp
chmod 700 ~borguser/tmp

Then create a wrapper script for Borg setting this variable. The result will be Borg using ~borguser/tmp for it’s private temporary directory, leaving /tmp alone, working nicely with the DSM v7.0 security design. I keep mine in ~borguser/.ssh and call it borg.sh. And, be sure it’s executable. Mine is like this:

adminuser@diskstation:/var/services/homes/borguser$ sudo cat .ssh/borg.sh
!/bin/sh
export TMPDIR=$HOME/tmp
/var/services/homes/borguser/borg serve --storage-quota 120G --restrict-to-repository /volume2/Backups/borg
adminuser@diskstation:/var/services/homes/borguser$ sudo ls -als .ssh/borg.sh
4 -rwx------ 1 borguser root 161 Nov 15 06:56 .ssh/borg.sh
adminuser@diskstation:/var/services/homes/borguser$

Finally, change ~borguser/.ssh/authorized_keys limiting the backup user to executing the new script.

command="/var/services/homes/backupuser/.ssh/borg.sh",restrict,from="192.168.0.23" ssh-rsa AAAA[...restofkeygoeshere...] remoteuser@remoteserver.example.com

Comments closed

My Photo Gallery Has Been Removed

Published November 15, 2022

Since 2001 or so I’ve been hosting most of my personal photos in a public gallery at nuxx.net/gallery (archive.org link to one of the first instances).

Today, ~21 years later, I’ve taken it down.

I’ve no single reason why; it’s basically a combination of these things:

Two years ago I made the photo gallery static, so nothing new was being added.
Keeping the gallery online costs $8/mo for the extra 80GB volume at Linode.
I previously used this gallery to share a lot of life online. I no longer do this in the same way.
Outside of social media, everything I publish online for the last few years has been via this blog.

The aforementioned static archiving of Gallery and backups with Borg made it easy to move hosting to my NAS at home. Now I can continue accessing the site myself while simplifying my public internet presence.

This feels like a huge shift, but the right one.

Comments closed

CycleOps (Saris) Hammer Rattle: Belt Tension?

Published December 22, 2020

I’ve had a CycleOps (now Saris) Hammer smart trainer since late 2017 and it’s been working great, but lately has been making a slight rattling sound during post-ride spindown just before the flywheel stopped. It felt fine during use so I kept riding, but then when putting out a couple hard, short efforts during high resistance periods (in slope mode via Zwift) it made a loud clank/bang sound. I would also sometimes get a rumbling feeling as if the notches on the belt weren’t smoothly engaging with the notched pulleys. Various posts online attribute this to a worn belt, so I opened up the trainer to take a look.

What I found led me to believe the issue is with the belt tensioner coming loose, not a worn belt. I suspect for many the belt replacement fixes the problem because the replacement process includes re-tensioning the belt.

Inside the trainer there is an idler pully whose tension is adjusted via a threaded rod on a spring. This rod is turned via a 5mm hex head accessible from the bottom of the trainer (without opening it). The threads on this rod seemed a bit worn, it seemed loose, and shaking it made a similar metal-on-metal sound to the rattling and clank that I’ve been hearing.

As the belt is a motor-type timing belt, it’s pretty unlikely that human-level output in clean conditions will stretch it much. Removing and inspecting the belt and all pulleys showed that everything was clean and free of damage, so I reassembled it, tightened the tensioner to compress the spring a bit and tension the belt, and rode the trainer a few times. After this there was no more rattle and I was unable to reproduce the clank/bang during hard efforts.

It seems the cause of the noise was the lack of tension on the threaded rod, spring, and idler and it likely came loose over time. The rattle was from the tension rod rattling as the as the heavy flywheel and main fan wheel and belt came to rest. The bang noise was from the sudden heavy pedal load vs. the strong resistance unit removing all belt pressure on the idler and it all slamming back together when my pedal stroke dropped off.

So, if you’re hearing this same sort of noise and rumbling, try using a long 5mm wrench to tighten up the tensioner a couple turns. If this works for you it’s much quicker (and cheaper) than replacing a belt.

This trainer is an original CycleOps Hammer, sometimes referred to as the H1, and the whole Hammer/H2/H3 family of trainers is now sold under the Saris name. (The CycleOps brand has been owned by Saris for years, but now all products are sold under the Saris name.)

To open a Hammer (H1) trainer you need a 1/18″ hex tool for the small screws, a T-30 Torx for the large screws. The cassette must first be removed, and after removing the screws the snug-fitting side panel pulls away. Belt tension can be adjusted with a 5mm hex and the fastener is accessible without opening the trainer. The stock belt is an MBL 150S5M930 and available for under $10 online. (Saris sells the replacement belt with the three required tools for $59.99 + shipping.)

Comments closed

Mail-Hijacking Malicious Profile on iOS

Published November 2, 2020

I was recently asked to look at a family member’s iPad because it was no longer sending email. Turns out that it had been set up to use an additional email account that steals copies of all their outgoing mail. Unfortunately, they didn’t notice until the attacker’s system stopped working and the iPad started showing an error message. Besides the irritating (or worse) spam they saw, their stolen emails could have been used for anything from spear phishing to accessing one’s online accounts, impersonating them, phishing others, delivering targeted spam, fake news / propaganda, etc.

So how did this get set up?

Apparently at some point this person installed the My Accurate Forecast app [1]. Included in this app was a Profile — or a set of settings for Apple devices — that added a second email account with address lazaroburst@my.minbox.email. This account was also set as the outgoing server for their Hotmail (Outlook.com) account.

This person would then have seen all messages in this account, with notifications just like their normal Hotmail email. Worse, everything they sent, from any email account, went to the attacker first. As it’s a separate email account, all the normal spam and malware protections from a normal email provider don’t apply… It’s a firehose of junk straight to their mailbox, with outgoing mail theft frosting on top.

This is bad because not only does it end up with them getting more spam, it allows the attacker to know exactly what they sent and to whom, and to modify those messages before delivering them to the intended recipients.

I think this was likely generated based on geolocated advertising, but it’s possible this individual was specifically targeted. The signed Profile had a name of “WEATHER ALERTS” a description of “Tap ‘Install’ above to get your local radar forecasts and weather alerts in 48062”, showing its intent to deceive; trying to make the normal Profile installation security alert — which is supposed to warn the user of a change to important settings — look like part of an application install.

I’m unsure when this first got installed, but judging by the the Profile signing certificate expiring on December 8, 2016 it was likely within a year or two prior. (Unfortunately I didn’t check the issuance date before deleting the profile.) The Profile which made these changes was signed by secure5g.com, an “advertising” company which has ties to minbox.email (the Unsubscribe link at the bottom of the page is a generic link to a minbox.email page).

A post from June 2018 on Medium, Unwanted Profiles Pop Up in iOS Devices, Inviting Spam and Malware, reports the same problem almost two and a half years ago. Curiously, the handful of other posts I read about this (ref: 1, 2) didn’t mention (or maybe didn’t notice) the outgoing server change? Perhaps because they only noticed before things broke, or maybe this iPad somehow ended up different? (It does seem that at least one other app: Daily Bible Verse, included similar email hijacking.)

Cleaning this up these settings was easy, just a matter of removing the malicious Profile, outgoing mail account, and setting the Hotmail account back to using the appropriate servers. But, who knows what damage was done with the theft of the sent mail and receipt of spammy stuff.

[1] The My Accurate Forecast website still shows screenshots of the app, but does not link to any app stores. It also no longer appears in the Apple App Store, implying that it’s been pulled out.

Comments closed

Bypassing Reolink SSID Length Limitation

Published October 14, 2020

I purchased a Reolink E1 Zoom camera for occasional around the house use. It turns out that my SSID, Smart Meter Surveillance Network is too long for their setup app. While the standard is 32 octets (32 ASCII characters) — and my SSID is exactly this — some things, such as the Reolink app, only accept 31 characters. In this case it pulled the SSID from my phone (the network in use) and then truncated it. †

So, I set out to find a workaround, and I did.

During setup the Reolink app walks you through scanning a serial number QR code on the camera, prompts for the wireless network info, and then generates a QR code and displays it on the mobile device’s screen. The camera is then pointed at the screen, this QR code is read, and the camera configures its WiFi settings based on the code.

I figured that maybe if I generated a new QR code with the correct info I’d be able to configure the camera with a longer SSID and it turns out that worked.

After a couple minutes of generating codes I found the configuration QR code is text, formatted as follows, with #### as the last four characters of the camera’s serial number:

<QR><S>ssid</S><P>password</P><C>####</C></QR>

Using the first free online QR code generator I could find, I created a new QR code with containing the following text:

<QR><S>Smart Meter Surveillance Network</S><P>notmyrealpassword</P><C>M77L</C></QR>

I reset the camera, had it scan the new QR code, and it connected to the wireless network. It worked! The camera was now on the wireless network and I was able to connect to it in the app.

There did seem to be a bit of quirkyness in the app, possibly because of the long SSID. It’s working fine with the desktop app, so all is good. It’s also really nice to now have a way of reconfiguring the camera without having to install and use their app.

† The standard maximum for SSIDs is 32 octets, or 32 ASCII characters. It appears some companies treat this as 31 characters, reserving the 32nd for the string termination character. Sort-of makes me wonder how I’ve been able to use this one for so long… It was fine with my old Apple AirPorts and I’ve had it running this way for couple years on Ubiquiti UniFi. Although it looks like the UniFi v6 UI now refuses to save changes with this SSID, so I guess I’m going to have to change it…

Comments closed

Archiving Gallery 2 with HTTrack

Published July 5, 2020

Along with the static copy of the MediaWiki, I’ve been wanting to make a static, archival copy of the Gallery 2 install that I’ve been using to share photos, for 15+ years, at nuxx.net/gallery. Using HTTrack I was able to do so, after a bit of work, resulting in a copy at the same URL and with images accessed using the same paths, from static files.

The result is that I no longer need to run the aging Gallery 2 software, yet links and embedded images that point to my photo gallery did not break.

In the last few years I’ve both seen the traffic drop off, I haven’t posted many new things there, and it seems like the old Internet of pointing people to a personal photo gallery is nearly dead. I believe that blog posts, such as this, with links to specific photos, are where effort should be put. While there is 18+ years of personal history in digital images in my gallery, it doesn’t get used the same way it was 10 years ago.

On the technical side, the relatively-ancient (circa 2008) Gallery 2 has and the ~90GB of data in it has occasionally been a burden. I had to maintain an old copy of PHP just for this app, and this made updating things a pain. While there is a recent project, Gallery the Revival, which aims to update Gallery to newer versions of PHP, this is based around Gallery 3 and a migration to that brings about its own problems, including breaking static links.

I’m still not sure if I want to keep the gallery online but static as it is now, put the web app back up, completely take it off the internet and host it privately at home, or what… but figuring out how to create an archive has given me options.

What follows are my notes on how I used HTTrack, a package specifically designed to mirror websites, to archive nuxx.net’s Photo Gallery. I encountered a few bumps along the way, so this details each and how it was overcome, resulting in the current static copy. To find each of these I’d start HTTrack, let it run for a while, see if it got any errors, fix them, then try again. Eventually I got it to archive cleanly with zero errors:

Gallery Bug 83873

During initial runs, HTTrack finished after ~96MB (out of ~90GB of images) saved, reporting that it was complete. The main portions of the site looked good, but many sub-albums or original-resolution images were zero-byte HTML files on disk and displayed blank in the browser. This was caused by Gallery bug 83873, triggered by using HTTPS on the site. It seems to be fixed by adding the following line just before line 780 in .../modules/core/classes/GallerySession.class:

GalleryCoreApi::requireOnce('modules/core/classes/GalleryTranslator.class');

This error was found by via the following in Apache’s error log:

AH01071: Got error 'PHP message: PHP Fatal error: Class 'GalleryTranslator' not found in /var/www/vhosts/nuxx.net/gallery/modules/core/classes/GallerySession.class on line 780\n', referer: http://nuxx.net/gallery/

Minimize External Links / Footers

To clean things up further, minimizing external links, and make the static copy of the site as simple as possible, I also removed external links in footer by commenting out the external Gallery links and version from the footer, via .../themes/themename/templates/local/theme.tpl and .../themes/themename/templates/local/error.tpl:

<div id="gsFooter">
{*
{g->logoButton type="validation"}
*{g->logoButton type="gallery2"}
*{g->logoButton type="gallery2-version"}
*{g->logoButton type="donate"}
*}
</div>

Remove Details from EXIF/IPTC Plugin

The EXIF/IPTC Plugin for Gallery is excellent because it shows embedded metadata from the original photo, including things like date/time, camera model, location. This presents as a simple Summary view and a lengthier Details view. Unfortunately, when being indexed by HTTrack, selecting of the Details view — done via JavaScript — returns a server error. This shows up in the HTTrack UI as an increasing error count, and server errors as some pages are queried.

To not have a broken link on every page I modified the plugin to remove the Summary and Details view selector so it’d only display Summary, and used the plugin configuration to ensure that every field I wanted was shown in the summary.

To make this change copy .../modules/exif/templates/blocks/ExifInfo.tpl to .../modules/exif/templates/blocks/local/ExifInfo.tpl (to create a local copy, per the Editing Templates doc). Then edit the local copy and comment out lines 43 through 60 so that only the Summary view is displayed:

{* {if ($exif.mode == 'summary')} * {g->text text="summary"} * {else} * <a href="{g->url arg1="controller=exif.SwitchDetailMode" * arg2="mode=summary" arg3="return=true"}" onclick="return exifSwitchDetailMode({$exif.blockNum},{$item.id},'summary')"> * {g->text text="summary"} * </a> * {/if} *    * {if ($exif.mode == 'detailed')} * {g->text text="details"} * {else} * <a href="{g->url arg1="controller=exif.SwitchDetailMode" * arg2="mode=detailed" arg3="return=true"}" onclick="return exifSwitchDetailMode({$exif.blockNum},{$item.id},'detailed')"> * {g->text text="details"} * </a> * {/if} *}

Disable Extra Plugins

Finally, I disabled a bunch of plugins which both wouldn’t be useful in a static copy of the site, and cause a number of interconnected links which would make a mirror of the site overly complicated:

Search: Can’t search a static site.
Google Map Module: Requires a maps API key, which I don’t want to mess with.
New Items: There’s nothing new getting posted to a static site.
Slideshow: Not needed.

Fix Missing Files

My custom theme, which was based on matrix, linked to some images in the matrix directory which were no longer present in newer versions of the themes, so HTTrack would get 404 errors on these. I copied these files from my custom theme to the .../themes/matrix/images directory to fix this.

Clear Template / Page Cache

After making changes to templates it’s a good idea to clear all the template caches so all pages are rendering with the above changes. While all these steps may be overkill, I do this by going into Site Admin → Performance and setting Guest Users and Registered Users to No acceleration. I then uncheck Enable template caching and click Save. I then click Clear Saved Pages to clear any cached pages, then re-enable template caching and Full acceleration for Guest Users (which HTTrack will be working as).

PANIC! : Too many URLs : >99999

If your Gallery has a lot of images, HTTrack could quit with the error PANIC! : Too many URLs : >99999. Mine did, so I had to run it with the -#L1000000 argument so that it’ll then be limited to 1,000,000 URLs instead of the default 99,999.

Run HTTrack

After all of this, I ran the httrack binary with the security (bandwidth, etc) limits disabled (--disable-security-limits) and used its wizard mode to set up the mirror. The URL to be archived was https://nuxx.net/gallery/, stored in an appropriately named project directory, with no other settings.

CAUTION: Do not disable security limits if you don’t have good controls around the site you are mirroring and the bandwidth between the two. HTTrack has very sane defaults for rate limiting when mirroring that keep its behavior polite, it’s not wise to override these defaults unless you have good control of the source and destination site.

When httrack begins it shows no progress on screen, so I quit with Ctrl-C, switched to the project directory, and ran httrack --continue to allow the mirror to continue and show status info on the screen (the screenshot above). The argument --continue can be used to restart an interrupted mirror, and --update can be used to freshen up a complete mirror.

Alternately, the following command puts this all together, without the wizard:

httrack https://nuxx.net/gallery/ -W -O "/home/username/websites/nuxx.net Photo Gallery" -%v --disable-security-limits -#L1000000

As HTTrack spiders the site it comes across external links and needs to know what to do with them. Because I didn’t specify an action for external links on the command line, it prompts with the question “A link, [linkurl], is located beyond this mirror scope.”. Since I’m not interested in mirroring any external sites (mostly links to recipes or company websites) I answer * which is “Ignore all further links and do not ask any more questions” (text in httrack.c). (I was unable to figure out how to suppress this via a command line option before getting a complete mirror, although it’s likely possible.)

Running from a Dedicated VM

I ran this mirror task from a Linode VM, located in the same region as the VM hosting nuxx.net. This results in all traffic flowing over the Private network, avoiding bandwidth charge.

Because of the ~90GB of images, I set up a Linode 8GB, which has 160GB of disk, 8GB of RAM, and 4 CPUs. This should provide plenty of space for the mirror, with enough resources to allow the tool to work. This VM costs $40/mo (or $0.06/hr), which I find plenty affordable for getting this project done. The mirror took N days to complete, after which I tar’d it up and copied it a few places before deleting the VM.

By having a separate VM I was able to not worry about any dependencies or package problems and delete it after the work is done. All I needed to do on this VM was create a user, put it in the sudoers file, install screen (sudo apt-get install screen) and httrack (sudo apt-get install httrack), and get things running.

Wrapping It All Up

After the mirror was complete I replaced my .../gallery directory with the .../gallery directory from the HTTrack output directory and all was good.

Comments closed

Using OsmAnd Offline Maps during Remote Rides

Published October 6, 2019

Many of the popular, long cycling events — or even just solo rides — require a GPS so that one can follow a route in order to navigate. While most of us have bike computers that will show the route, most folks don’t have a GPS with the kind of detailed base maps that become useful for detours, emergencies, or just as an additional tool… Much less something that works when outside of cellular coverage areas.

For this, I recommend you look at offline maps using OsmAnd.

I have (and love) the Garmin Edge 130 as it’s perfect for my riding and training. It’s been great for displaying recording data and following routes from the unmarked 100 mile versions of The Crusher and Barry-Roubaix, to augmenting signage in Marji Gesick and following a route through remote parts of Batchawana River Provincial Park. But occasionally I need a bit more; something to help find a detour around a flooded road, a quick way back to the car in a rainstorm, or just a sanity-check of which branch to take at a fork on the road.

Sure, I could buy a super-high-end bike computer that features detailed base maps, but I rarely need these and don’t want the physically large computer on my bike. So I use my phone, running OsmAnd, loaded with offline maps that display without requiring a data connection. Coupled with putting one’s phone in Airplane mode (turns off the cellular and WiFi radios and saves battery — yet GPS still works) a modern phone will typically get days of battery life while sleeping, with maps and a route quick to access.

Whenever I actually need to see this map I simply pull it out of my pocket or bag, wake it up, get what I need from the map, and carry on. No screwing around with a small touch screen and funky zooming on a Garmin or Wahoo. (I believe that, to date, neither Garmin nor Wahoo have made map perusal as straightforward as it is on a basic mobile phone.)

This is also a great way to get maps on your phone when traveling somewhere costly for cellular data, like during a day-trip to Canada.

Whether you’re doing Marji Gesick or The Crusher or just following a route found on a cycling club’s website, I recommend augmenting your navigation with offline maps in OsmAnd by doing the following:

Visit the OsmAnd site and follow the links to your phone’s store to install the software.
Get a copy of the GPX file on your phone; for Android phones this is often as simple as visiting the download website and downloading the file.
In OsmAnd‘s menu, pick Download Maps and download maps for the regions you want. I personally have Standard Maps, Contour Lines, and Hillshades for Michigan installed all the time, and then if traveling will install for other states/provinces/countries.
Once the offline maps are installed, pick Configure map from the menu, then Map source, and ensure Offline vector maps are selected. The default, OsmAnd (online tiles), is an online map source that requires data.
To display a particular route, pick GPX files…, Add More…, and then browse your phone to add the downloaded GPX file. Ensure this file is enabled (checked), and then go back to the map screen.
You’ll now see the route overlaid on offline maps, easy to zoom in and out of and find your way around. Try turning on Airplane Mode and see that it all still works.

Once I have everything set up, here’s some tips about how I actually use OsmAnd while out on my bike:

If in an area with minimal or no cellular service, put the phone in airplane mode so the battery doesn’t get used up while it searches for a signal.
Turn off all lock features on my phone (fingerprint / pattern / PIN unlock), so I can wake it up with just a press of the power button.
You can use this at the same time as Strava, and Strava generally works in Airplane mode as well.
If I’m at a spot where I’m not quite sure which way to turn (based on the single line on my Edge 130), use the map to figure out which branch to turn on. Even the remotest of fire roads and two track are often mapped.
If I think I might have missed a route sign (say, in Marji Gesick) I’ll use OsmAnd to see if I’m still on route, and to get back if needed.
Look for other nearby roads or trails if I need to detour or shortcut back. I had to do this yesterday during the Founders Fall Fondo as part of the 62 mile Barry-Roubaix route was flooded.
Many small, old, sometimes abandoned, yet somewhat navigable roads are still listed in the OpenStreetMap (OSM) data that OsmAnd uses. While these aren’t great for cars and won’t show up on things like Google or Apple Maps, they are often quite useful when on a bike. Look to these for exploring / finding new routes.
Many trails open to mountain bikes are included in the OSM data. This includes all CRAMBA trails and most trails in the NTN and RAMBA areas. (Note: Trailforks is a better source for discovering MTB trails themselves.)
Periodically update maps to get new OSM data. As roads and trails change, maps get updated.
Pay For OsmAnd+ Live ($5.99/year) to get access to more-frequent, automatic map updates and support the open mapping community.

While this is just a basic overview of using OsmAnd and how I use it when cycling; it does a whole lot more including recording routes, navigation, editing map data, sharing location between contacts, and more. Give the online Help and Features a look for more information.

Comments closed

Torklift EcoHitch 2″ Hitch for my 2015 Outback

Published January 7, 2019

Back in 2015 when I purchased a Subaru Outback, I ordered it with a OE hitch. At the time it was the best (and thanks to IMBA discount the cheapest) option for mounting a bike rack on the back of the car. Due to load limits on what the vehicle can tow there was only a 1-1/4″ receiver available, and while this fit the 1Up USA rack previously used on my Civic it just didn’t provide the stability of a 2″ receiver.

Fast forward to late 2018 and, out of the blue, I was contacted by Torklift, via Reddit, where they offered to send me their Torklift EcoHitch 2″ receiver hitch in exchange for providing feedback to their designers and engineers. After some email back and forth this was all set up, and a few days ago I received the rack: Torklift 2015-2018 Subaru Outback EcoHitch, part number x7266.

Beyond the 2″ receiver, the biggest difference between the EcoHitch and OE hitches is how they mount. While the EcoHitch sandwiches between the bumper beam and the chassis, held in place by the bumper’s eight fasteners (photo 1, 2), the OE hitch both replaces the bumper beam and mounts inside of the frame rails (photo). While it appears that the OE hitch may be a bit more resistant to very high tongue weights, the Torklift seems plenty solid and is made of thicker material, so I’ve got no concerns with the differences. The other significant difference is that the Torklift EcoHitch doesn’t come with a wiring harness, but only ever having bike racks on my car this doesn’t really affect my usage. When the OE hitch was installed on my Outback this included a wiring harness along with a nifty mount to hold it next to the receiver (photo). I considered adapting this mount to the EcoHitch, but instead took some inspiration from Torklift Central’s (frankly overpriced) Eclipse 4-Flat Plug and fit a neodymium magnet to the cable (photo), then stuck the magnet to the back side of the receiver.

Needing to remove the OE hitch my install was bound to be a bit more complicated than someone who is putting the EcoHitch on a new vehicle, but even this ended up being pretty straightforward.

Whole rack, rear view, with installation instructions and parts bundled to the center.

Unboxing the rack revealed a well-packed (photo), nicely powder coated (photo), solid rack that ships with an instruction manual, the eight nuts (for attaching it to the bumper mount bolts), and a piece of rubber trim gasket to cover the cut edges on the underside of the bumper cover (photo).

Besides the printed instructions (which I wish Torklift posted as a PDF on their website), there are two instructional videos (1, 2) which show the installation on everything from 2015 to 2018 models of the vehicle. These do a good job of showing the basics of the installation, but they gloss over what I think is the most difficult part; removing the bumper cover:

Removing the bumper cover itself works best if you push up on the sides from the bottom, then pull out the at top. Going off of just Torklift’s instruction I first thought to pull straight outward, but this wasn’t working well and — per Subaru — can risk breaking retention clips. Thankfully I have access to Subaru’s service manual which described pushing up from the bottom, and with this technique the cover came off easily.
My vehicle has splash guards installed, which had to be removed before all the bumper clips could be accessed. These guards came off via six screws and four push-lock plastic clips, but I had to use a right-angle screwdriver to fit around the tires. (There is a note in the Torklift instructions mentioning these may need to be removed.)
Three out of the myriad push lock clips used to hold the bumper and splash guards on broke during removal. This is likely due to my Outback being nearly four years old. Thankfully replacements are readily available online and at dealerships and can easily be identified via Subaru Parts Online.

For this installation Torklift instructs that the hitch be fitted, then the bumper beam reinstalled. However, during the install of the OE hitch, the bumper beam and foam energy absorber is removed and discarded. Since my Outback came with the OE hitch installed, I didn’t have these pieces. After consulting with both Torklift and some auto engineer friends we concluded that the Torklift hitch will be fine without the bumper beam.

Despite this, I contacted my local dealer (Sellers Subaru) and to my surprise they gave me discarded/scrap bumper beam from an Outback where they’d recently installed a hitch. It’s maroon, but being hidden the color doesn’t matter. With the purchase of a replacement foam energy absorber ($79.95 MSRP) I was all set part-wise to remove the OE hitch and install the EcoHitch to spec. While this likely wasn’t needed, it makes me feel better about the overall install. (Photos 1, 2)

Removing the OE hitch was a bit of a hassle, as getting to two of the hitch bolts required removing the muffler and heat shield. (Another plus for the EcoHitch.) Thankfully none of the bolts were seized and removal this went smoothly. Before long I also had the OE rack completely removed, exhaust re-fitted, and it was time to install the EcoHitch.

Foam cushion over the bumper beam, and hitch installed.

At this point — just after bumper cover removal on a normal install — installation is very straightforward: folding a thin metal bumper cover mounting tab up against the body, bolting the hitch and bumper back on, torquing some nuts, and cutting a notch out of the bumper cover. After this it’s just a matter of putting the the bumper cover, tail lights, splash guards, and some plastic covers back in place.

From the OE hitch I already had a notch cut in my bumper cover, but it was pretty easy to cut the wider notch required by the EcoHitch. The OE hitch uses a longer notch than the EcoHitch, and thankfully Torklift provides enough trim to cover the extra-large notch present after a bumper’s been cut for both (photo). The cutting itself went easily, as the plastic is pretty soft and can either be scored and folded or cut with anything from a rotary cutter to a jigsaw.

During installation one of the bumper cover mounting tabs is folded out of the way and the mounting hole cut out of the bumper cover during the notching, which eliminates one of the points where the bumper mounts to the chassis. While initially concerning, this part of the cover rests on the top of the receiver after everything’s put back together, so this lack of retention point isn’t a problem.

With everything put back together I’m currently quite happy. The TorkLift EcoHitch looks good and sits nice and close to the underside of the bumper; I don’t think it could go much higher. While the OE hitch had 14.5″ of ground clearance, this drops to 13.5″ with the EcoHitch, an acceptable change for adding 0.75″ to the receiver’s height. This hitch is a great choice for either new installs on an Outback or retrofitting an OE hitch to a 2″ receiver.

I particularly like how the powder coating has a thick, textured look to it, and the end of the receiver has a nice TORKLIFT logo. The whole unit looks like it’ll hold up nicely to Michigan winters (and road salt). Now to wait for a new, matching black 1UP USA 2″ Super Duty Single rack to arrive!

Click here or on the large photo above to see a complete album of photos from the install.

Trailforks, Pinkbike, and OSM

Published April 10, 2016

For years I’ve been pretty enthusiastic about OpenStreetMap (OSM) and using it to map trails (MTB and otherwise). While there are a bunch of other ways to map trails online (Google Maps, MTB Project, Trailforks) I have stayed away from contributing to them because of the one-way nature of submissions; your contributed data gets locked behind their license. While MTB Project and Trailforks both claim to allow some manner of reuse of data, it’s nothing as useful as OSM‘s Creative Commons (CC) based licensing. Effectively being the Wikipedia of GIS makes it extremely useful for those of us who want to both contribute data and build open maps on the larger set.

Then suddenly last night I read this article on Pinkbike discussing how they took OSM data, parsed it to highlight mountain biking routes, and are now using it as the base map for their Trailforks mapping site. They built a tool on top of the open data and made something great.

This is really, truly excellent.

This sort of reuse of public, open data in OSM is the exact reason why I contribute to it. The folks at Pinkbike / Trailforks have taken a useful set of data from all over the world, processed it, and made something good. This would not have been possible with the data locked up in Google, MTB Project, or even the stuff contributed directly to Trailforks.

I look forward to where this’ll go. The Pinkbike article mentions that they’ll be reimporting the data a little different in the future, and talks about how they are going to have another article about tagging to better support Trailforks. While OSM has some minimal standards for MTB tagging (eg: mtb:scale:imba) I look forward to a bit more de-facto standard around this.