Press "Enter" to skip to content

nuxx.net Posts

Easy Carpet Spikes for iMovR Freedom Base

I recently purchased an iMovR Energize corner standing desk which came with the Freedom base. It works well, but had a bit of a wobble when placed on the relatively-thick carpet in my office. Because the leveling legs are relatively wide (35mm) they’d sit on top of the carpet and the desk didn’t have great support.

To solve this I picked up four M8-1.25 x 25mm hex head screws from Home Depot and fitted them in place of the leveling feet. This resulted in ~20mm tall, narrow feet sticking down off the legs, pressing firmly through the carpet to the wood floor below, and no more wobble.

This is the same principle as carpet spikes, used to for speakers and other tall/narrow cabinets, to make them more stable on soft carpet by pressing through the carpet to the hard floor below. (Carpet spikes, for speakers, have all sorts of other acoustic isolating purposes which sometimes border on audiophile woo, but increased physical stability is an easily demonstrated effect.)

Comments closed

Bontrager Line Dropper Post Failure Mode + Repair

The Bontrager Line Dropper Seatpost, as fitted on Kristen’s Trek Fuel EX 9.8 Women’s (mirror) is a quality dropper, and I’m particularly impressed with the way built-in sacrificial parts fail when the saddle is hit hard from the side. Over the summer Kristen has had a couple crashes which, due to hard impacts on the side of the saddle, damaged the dropper. After the first crash the saddle (and inner part of the dropper) would turn easily to one side, and after the second the saddle had a bunch of side to side play, and could be turned to the side fairly easily.

On this dropper there are two plastic keys that slide in grooves in the outer tube as the saddle moves up and down. They keep the saddle from moving to the side, are designed to fail when the saddle is hit hard from the side. By using a sacrificial part like these plastic keys, Trek/Bontrager’s designers have a dropper which works well, but only costs a few dollars to repair after a damaging crash.

The key set, part number 572184 and $5.99 MSRP, is replaced by removing the seatpost from the bike, unscrewing the bottom of the post by hand, then unscrewing the retaining ring at the top of the post with a strap wrench. Sliding off the outer tube reveals the keys, which can be popped out with a pin or a razor blade. Wiping everything down, fitting the replacement keys in the groove, lubing with Slickoleum, then putting everything back together is all that’s needed to repair the dropper to like-new condition.

The photo above shows a pair of damaged keys, along with the plastic shavings cleaned out of the dropper after a failure. The rounded edges on the keys show where they fail when overloaded, and the shavings are the remains of the once-sharp edges.

I’m really happy with this dropper. It works well, it’s overall pretty cheap, is easy to disassemble to repair after a crash, and replacing the main cylinder should be just as easy, whenever it comes time for that.

Comments closed

Suggested First Rides in Marquette

Here’s a quick list of easy to follow mountain bike routes in the Marquette area, as Trailforks routes. Fun, accessible routes friendly to all bike types, from rigid fatbikes to squishy trail bikes, perfect for getting you started riding in the Marquette area. Each loop will take most riders an hour to and hour-and-a-half each, excluding stops, and are excellent on their own or as a basis for exploring other trails.

These trails are all built and maintained by the Noquemanon Trails Network and are constantly being improved and expanded. Without their work you wouldn’t have these great trails to ride, and without donations they can’t exist. Click here to send some money their way to keep these trails great. (Sign up for a full NTN membership here.)

Keep in mind all these trails are two way and quite popular in both directions. Be nice, say hi, yield appropriately, and let others know how many more folks are behind you.

NTN North Trails (from BLP Trailhead)

Kristen’s Favorite Loop: One of the best ways to get started with riding North Trails. An easier route than the South Trails, but by no means boring, this route includes views of the awesome Forestville Dam and Falls, Wright Street Falls, Forestville Basin, the penstock (large wood pipe). Climbing is gradual but sustained, as are the descents, with just enough rock sprinkled in to make things fun.

NTN South Trails (from South Trailhead)

Green / Morgan Creek Loop (Clockwise): Perfect intro to the South Trails, with rolling climbs and descents, riding past and over waterfalls. Scatterings of rocks and roots are all over, but nothing difficult; the perfect everything trail. Follow the green signs.

Red / Pioneer Loop (Clockwise): Begins with the Benson Grade Access Road climb, then starts with a relatively flat but slightly rocky single track before snaking it’s way along a beautiful brook and mildly rocky trails along with views of Lake Superior. A bit more technical than Green, but another great trail to get started on. Follow the red signs.

Gorge-ous to Blue: A step above the Red and Green loops, this heads downhill on the incredibly scenic Gorge-ous trail (part of the Yellow route) and loops back to the trailhead using portions of the Blue loop. Gorge-ous is mostly smooth dirt, with a handful of rocks and roots, and some decent (for Michigan) exposure. Blue, the oldest route in the system, is considerably rougher and has some challenging climbs, but is still a lot of fun. Starts by following Grom (Purple Signs) to Gorge-ous (Yellow Signs) and then continues on Forget-Me-Not (Blue Signs) after reaching the Carp River.

(If you want more information, check out my longer post, Marquette Mountain Biking for Trolls, which gives downstate Michigan folks pointers on getting started riding in the Marquette area.)

Comments closed

Crankbrothers Eggbeater Pedals Wing Wear

Crankbrothers Eggbeater and Candy pedals have been my go-to for years, performing admirably year-round.

All Crankbrothers pedals require periodic maintenance in the form of the Pedal Refresh Kit, which replaces the bushings and bearings and gets them spinning like new. The cleats — made of brass — wear out as well and require replacement roughly once a season, along with the required-for-carbon-soles Shoe Shields.

Unfortunately, there is one kind of wear which puts the pedal near the end of its life: pedal wing wear. Above you can see the pedal wings on the Eggbeater 3 pedals from my Specialized Camber, which worn to a point.

The wings are normally box shaped, providing a nice even load on the cleat and sole of the shoe. After wearing to a point the cleat and Shoe Shield wear are accelerating, to the degree that my cleats have worn out mid-year instead of late autumn.

With all this wear put together, pedaling my Camber felt a bit vague and I’d get squeaking and clicking sounds from the pedals when putting down a lot of steady power. Flipping the pedals 90° to change up the engagement would help, as some of the wings are worn less than others, but it’s now time for the pedals to go.

(Instead of replacing with new Crankbrothers pedals I’m giving Shimano PD-M8100 (Deore XT M8100) SPD pedals a go. I’m hoping the larger contact area between the cleat, pedal, and shoe lugs will help with some foot pain problems I’ve had during longer rides. I also expect cleat replacement will be less frequent, costing less long-term. I have concerns about how well SPDs will clear snow and/or permit very fast exit when suddenly stopping on technical sections, so I won’t be selling off my remaining Eggbeaters any time soon.)

Comments closed

Shimano Hydraulic Brakes May Self-Contaminate Due To Residual Oil In Bleed Nipple

It seems the design of the bleed nipple on Shimano hydraulic disc brakes may result in contamination of the brake pads if extra care isn’t taken to clean residual oil from inside the nipple after bleeding the brakes. Newer Shimano hydraulic calipers, such as the Shimano BR-M7100 (SLX), have this nipple facing downward when installed on most bikes which seems to exacerbate the issue.

After following the Shimano brake bleed procedure and disconnecting the hose, the nipple will still contain about 0.06 mL of brake fluid, roughly a full drop, closed only by a snap-fit rubber cap. (See exploded view, inside of nipple is ~20mm x ~2mm ⌀.) On many brakes, including the BR-M7100 when mounted to a fork or seatstay, the nipple points downward and the residual oil inside slowly weeps out, wetting the outside of the cap and the caliper. Particularly after mixing with dust and forming an oily paste this can fling to the rotor or pads, contaminating the pads, leading to poor performance and noise.

On other Shimano brakes, such as the the BR-M8000, the bleed nipple is on the other end of the caliper. These point up and don’t seem to weep residual oil as readily. However, because bikes are stored and transported in a variety of positions, much less bounced all over the place while riding, any oil in the nipple can cause escape.

To avoid self-contamination it’s necessary to remove all residual oil from inside of the bleed nipple after a bleed. This can be done by twisting the corner of a paper towel into a point, shoving it into the nipple, and blotting the oil up. A couple iterations of this and thorough cleaning of the caliper and inside of the nipple with isopropyl alcohol seems sufficient. After doing this the nipple caps on our bikes have remained dry.

I came to this realization after a handful of dusty rides on the Timberjack when I noticed this cap had a bunch of dark oil-soaked dust on it. A quick check showed the inside of the cap was quite oily. There was also a thin film of oil wicking on to the bleed nipple and caliper body. As the bike is nearly new the brakes had recently been bled and the outside of the calipers thoroughly cleaned, it all fit together. (I suspect this may have led to the contamination problems I had earlier with the pads, although those pads themselves seemed bad from the get-go.)

On Kristen’s fatbike — a 2018 Specialized Fatboy Carbon Comp which received M7100 SLX brakes to replace the failed SRAM Level TLs but has only been kept upright since the brake install — the front brake whose nipple points down had oil in the cap. The rear brake, mounted to the chainstay and pointing the nipple up, was dry, but still had visible oil in the nipple.

This could also explain a mysterious fouled-front-brake problem on my Warbird, whose BR-R7070 (105) calipers have a downward facing bleed port on the front and upward facing on the rear. This was fixed with a sanding of the rotor and pad replacement, but I could not find a source of oil and the system seemed sealed. I now believe residual oil migration past the rubber cap, after I bled the brakes following a fork replacement, fouled the pads.

Comments closed

Preferred Bicycle Lubricants

Here is a list of the lubricants I use for bicycles and a few notes about each one.

General Grease
Park Tool PolyLube 1000 (PPL-1, Tube)
Use for general greasing. Threads of fasteners, coating bearings before installation, etc. This is a go-to grease that gets used on everything unless there’s a specific need for something special.

Chain Lube
ProGold ProLink Chain Lube
Use for all chain lubing purposes. As this lube is a heavier oil in a lighter carrier, I use the following process: Wipe chain with dry paper towel to remove dirt and old lube. Wipe chain with alcohol-soaked paper towel if it’s particularly dirty. Apply one drop to each roller on the inside of the chain. Turn crank backwards for 10-15 seconds to ensure lube is well distributed. Use a new dry paper towel to wipe off the outer plates of the chain (lube does nothing here). Let sit for a while, perhaps overnight, before riding so the the volatile compounds in the lube can evaporate leaving only the useful stuff. It’ll pick up less dirt this way, too.

Waterproof Grease
PEAK Synthetic Marine Grease (branded as Advance Auto Parts Marine Grease)
Used whenever a heavy, highly water resistant grease is needed. I use this on the lower bearing on headsets, bottom bracket spindles, car hitch racks. Use with caution as this grease attracts dirt, thickens, and migrates pretty easily and thus isn’t good for basic lubricating. (Any standard marine grease will work in place of this, the Advanced Auto Parts version was the cheapest when I bought some.)

Anti-Seize
Permatex Copper Anti-Seize Lubricant
Anti-seize is a grease with metal powder in it, used to inhibit galvanic corrosion when dissimilar metals are in contact. Instead of the original parts corroding the small metal flakes in the grease will corrode, prolonging the life of the parts and preventing seizing. I mostly use this on titanium frames as it’ll quickly corrode aluminum parts (such as headset cups, bottom brackets, seatposts, and mounting screws) but also use it on steel and aluminum frames when installing press-fit headsets and threaded bottom brackets, as a preventative measure.

Suspension Grease
Buzzy’s Slick Honey / Slickoleum / SRAM Butter
All three of these products are the same thing. It’s ideal for lubricating anything that slides or is suspension-related. Also works great on dropper posts. It’s also an ideal lube for Hope freehubs.

Small / Fine Parts
Tri-Flow Superior Lubricant (Drip Bottle)
This is a very thin lube which carries PTFE (Teflon). Perfect for lubricating small pivot points such as derailleurs and shifters.

DT Swiss Ratchets
DT Swiss Special Grease (Red, HXTXXX00NSG20S)
DT Swiss hubs, with star ratchets, specifically call for a tacky, yet somewhat thin, red grease which DT Swiss calls Special Grease. So little is used on each cleaning that a small container, one of which comes with every replacement ratchet set, will last for years.

Friction Paste
Finish Line Fiber Grip / Park Tool SuperGrip (SAC-2)
Sometimes things slip when you don’t want them to (eg: seatposts, bars) or you want to add extra grip without torquing tighter. Friction paste, a light grease with sandpaper-like grit in it, is perfect. It’s common to use this on the handlebar clamp part of a stem to ensure the bar doesn’t move, on seatposts in carbon frames, etc. Never use this on anything which is supposed to move, and be aware that it’ll abrade the clamped surfaces of whatever you apply it to.

Spoke Nipple Lube / PTFE Paste
ULTRA Tef-Gel
When building wheels I lube the spoke threads with ULTRA Tef-Gel, which is a PTFE (Teflon) paste. Designed for use on saltwater-exposed fasteners, this is an incredibly tenacious anti-corrosive that keeps spokes and nipples from binding together doubles as lubricant during assembly. Use ensures they’ll still be turnable after years of year-round exposure. This also works well for installing press-fit bottom brackets which call for PTFE paste.

Comments closed

Simple PAC File Pilot Testing (including WPAD)

In a network that’s isolated from the public internet, such as many enterprise networks, proxy servers are typically used to broker internet access for client computers. Configuring the client computers to use these proxies is often done via a Proxy Auto-Config (PAC) file, code that steers requests so traffic for internal sites stays internal, and public sites go through the proxies.

Commonly these PAC files are made available via Web Proxy Auto-Discovery Protocol (WPAD) as well, because some systems need to automatically discover them. Specifically, in a Windows 10 environment which uses proxies, WPAD is needed because many components of Windows (including the Microsoft Store and Azure Device Registration) will not use the browser’s PAC file settings; it’s dependent on WPAD to find a path to the internet.

WPAD is typically configured via DNS, with a hostname of wpad.companydomain.com (or anything in the DNS Search Suffix List) resolving to the IP of a webserver [1]. This server must then answer an HTTP request for http://x.x.x.x/wpad.dat (where x.x.x.x is the server’s IP) or http://wpad.company.com/wpad.dat with a PAC file, with a Content-Type of x-ns-proxy-autoconfig [2].

Because WPAD requires DNS, something which can’t easily be changed for a subset of users, putting together a mechanism to perform a pilot deployment of a new PAC file can be a bit complicated. When attempting to perform a pilot deployment engineers will often send out a test PAC file URL to be manually configured, but this misses WPAD and does not result in a complete system test.

In order to satisfy WPAD, one can set up a simple webserver to host the new PAC file and a DNS server to answer the WPAD queries. This DNS server forwards all requests except for those for the PAC file to the enterprise DNS, so everything else works as normal. Testing users then only need to change their DNS to receive the pilot PAC file and everything else will work the same; a true pilot deployment.

Below I’ll detail how I use simplified configurations of Unbound and nginx to pilot a PAC file deployment. This can be done from any Windows machine, or with very minor config changes from something as simple as a Raspberry Pi running Linux.

[1] WPAD can be configured via DHCP, but this is only supported by a handful of Microsoft applications. DNS-based WPAD works across all modern OS’.

[2] Some WPAD clients put the server’s IP in the Host: field of the HTTP request.

DNS via Unbound

Unbound is a DNS server that’s straightforward to run and is available on all modern platforms. It’s perfect for our situation where we need to forward all DNS queries to the production infrastructure, modifying only the WPAD/PAC related queries to point to our web server. While it’s quite robust and has a lot of DNSSEC validation options, we don’t need any of that.

This simple configuration forwards all requests to corporate Active Directory-based DNS’ (10.0.1.2 and 10.0.2.2) for everything except the PAC file servers. For these, pacserver.example.com and wpad.example.com, it’ll intercept the request and return our webserver’s address of 10.0.3.25.

server:
interface: 0.0.0.0
access-control: 0.0.0.0/0 allow
module-config: "iterator"

local-zone: "wpad.example.com." static
local-data: "wpad.example.com. IN A 10.0.3.25"

local-zone: "pacserver.example.com." static
local-data: "pacserver.example.com. IN A 10.0.3.25

stub-zone:
name: "."
stub-addr: 10.0.1.2
stub-addr: 10.0.2.2

This configuration allows recursive queries from any hosts, but by specifying one or more subnets using access-control clauses to you can restrict from where it is usable. The stub-zone clause to send all requests up to two DNS’. If these upstream DNS’ handle recursion for the client, the forward-zone clause can be used instead.

PAC File via nginx

For serving up the PAC file, both for direct queries and those from WPAD, we’ll use nginx, a powerful but easy to use web server to which we can give a minimal config.

Put a copy of your PAC file at …/html/wpad.dat under nginx’s install directory so the server can find it. (There is great information on writing PAC files at FindProxyForUrl.com.)

This simple configuration will set up a web server which serves all files as MIME type application/x-ns-proxy-autoconfig, offering up the wpad.dat file by default (eg: http://pacserver.example.com) or when directly referenced (eg: http://10.0.3.25/wpad.dat or http://wpad.example.com/wpad.dat), satisfying both standard PAC file and WPAD requests.

events {
worker_connections 1024;
}

http {
default_type application/x-ns-proxy-autoconfig;
sendfile on;
keepalive_timeout 65;

server {
listen 80;
server_name localhost;

location / {
root html;
index wpad.dat;
}
}
}

Putting It All Together

With all the files in place and unbound and nginx running, you’re ready to go. Instruct pilot users to manually configure the new DNS, or push this setting out via Group Policy, VPN settings, or some other means. These users will then get the special DNS response for your PAC and WPAD servers, get the pilot PAC file from your web server, and be able to test.

Comments closed

Salsa Timberjack: Fixing a Mistake

After building up the All-City Electric Queen and riding it a handful of times, it just didn’t feel… right. Standing and handling the bike was fine, but seated pedaling — especially when climbing or just after sitting back down — felt quite off. Turns out the problem was the 71° seat tube angle on the frame coupled with my shorter femurs; I simply can’t get the saddle far enough forward on one of these frames.

Unfortunately, the only good solution was to get a different frame. The Salsa Timberjack was my original choice for this hard tail trail bike build, but I got excited by the idea of a steel bike, loved the paint on the Electric Queen, and glossed over the seat tube angle. By the time I realized I needed a new frame the stand-alone black frames were no longer available. Fortunately, over at the excellent Sports Rack Marquette, Evan had some new frames from complete bikes available, and I was able to get a beautiful gloss teal frame from a 2019 Timberjack Deore 27.5+ from them.

Besides matching my wants geometry-wise, the frame is a great choice because all parts except for the headset swapped over, and the frame came with a headset. While the stock Cane Creek 10 is a lower end part, which lacks sealing on the top bearing cover and has a plastic compression rings and crown races and black oxide bearings, it works and will be fine for a while. The fork was already fitted with a higher end matching crown race, and I have a Cane Creek Hellbender 70 headset ready swap in once the bearings and compression ring start to go.

One downside to the Timberjack vs. Electric Queen is that I’ll no longer have a rigid fork for the bike, but if I really want one the Firestarter 110 Deluxe is a perfect match. The top tube on this bike is also a little bit tall, as it’s also designed for bikepacking and fitting a top tube bag, but it’s plenty comfortable to ride and I love all the bottle cage options.

To round out the build and get the colors nice I ordered some new fork decals from Slik Graphics. Unfortunately, I screwed up and ordered the decals for the Factory-series forks, so while it looks good, I technically have the wrong upper logo on the fork lowers. I’ve since ordered another set with the proper Performance decals for the upper, and am waiting for them to arrive. Since this order was placed Slik became involved in a dispute with Fox, so I’m hoping to receive the updated decals. Even if they don’t arrive, at least the colors are right on the fork. I could even remove the upper decals and have it still look good.

When finishing up the build I ran into a significant problem with the brakes: squealing and vibration. Due to part availability I’d purchased the calipers and levers as a non-retail / meant-for-complete-bikes / likely grey market set from a well-known eBay seller, ronde-cycling. I was never able to get them bedded in properly, and after a few rides they began squealing horridly and shuddering under hard braking. This seller offers different pads options with the brakes, and I began to suspect they handle the pads with each brake set sale, did so poorly, and contaminated the pads before they got to me.

I tried the normal recommendations of cleaning everything, sanding the pads and rotors, and even baking the pads in the oven, but on each bed-in procedure they’d begin squealing again. Resolution a set of new J04C pads and a bed-in and now the brakes are working great. At ~$50 for a new set of pads this really added to the cost of the brakes, but at least they are now working.

Final build, with water bottle cages, pedals, and computer came out to right around 27 pounds. And, it fits! Since building it I’ve put over 180 miles and nearly 16 hours on the bike, haven’t touched the geometry, and I’m really happy with the result. It’s exactly what I wanted; a high quality hard tail trail bike.

Full details below:

Frame: Salsa Timberjack (Large, Teal, 2019)
Fork: Fox 34 Step-Cast (Performance, FIT4 damper, Black Upper Tube Finish, 120mm, 51mm offset, 15QR)
Fork Decals: Slik Graphics Fox 34 Step-Cast Factory Style Decal Kit / Fox 34 Step Cast Performance Elite Decal Kit (Color 1: Medium Grey, Color 2: Dark Grey, Finish: Matte)
Headset: Cane Creek 10 (Black, ZS44/ZS56)
Crankset: SRAM X1 1400 GXP
Bottom Bracket: SRAM GXP (Black)
Chainring: SRAM X-SYNC 2 (32t, steel, Direct Mount, 3mm / Boost)
Derailleur: SRAM GX Eagle
Shifter: SRAM GX Eagle
Shift Cables / Housing: Shimano Bulk
Cassette: SRAM XG-1275
Brakes: Shimano SLX M7100 (Levers: BL-M7100, Calipers: BR-7100)
Brake Pads: Shimano J04C (Finned, Metal)
Front Rotor: SM-RT86-L (203mm)
Rear Rotor: SM-RT86-M (180mm)
Front Brake Adapter: SM-MA-F203P/P (160mm Post to 203mm Post)
Rear Brake Adapter: Shimano SM-MA-R180P/S (IS to 180mm Post)
Stem: Salsa Guide (+6°, 60mm)
Bar: Salsa Salt Flat (750mm)
Wheels: Industry Nine Trail S Hydra 28H (29″)†
Tires: Maxxis Rekon (29 x 2.4″, 3C/EXO/TR)
Seatpost: Fox Transfer Performance Elite (2020, Black, 125mm, 30.9mm, Internal)
Dropper Lever: Wolf Tooth ReMote Light Action (Black, 22.2mm Clamp)
Seatpost Collar: Salsa Lip-Lock (Black, 35.0 mm)
Saddle: Specialized Power Expert (143mm)
Pedals: Crank Brothers Eggbeater 3 (Green, from Blackborow)
Grips: ESI Extra Chunky (Black)
Bottle Cages: Specialized Zee Cage II (Black Gloss, 1x Left, 2x Right)
Computer: Garmin Edge 530, Garmin Speed and Cadence Sensors (v1), Best Tek Garmin Stem Mount
Bell: Mirracycle Original Incredibell (Black)
Derailleur Hanger: QBP FS1373
Frame Protection Tape: 3M 2228, McMaster-Carr UHMW PE
Cable Rattle Prevention: Frost King EPDM Weatherseal (V25A, slipped over dropper housing)

Comments closed

CRAMBA Trails Outline Poster from OSM Data

Finding myself a little bored, I put together a poster (11″ x 17″) showing outlines of the CRAMBA-supported trails on one overview. (Link)

This ended up being more popular than I expected, with a handful of people wanting to know how I did it, so I’ll detail the steps here:

  1. Ensure that all the trail routes are in OpenStreetMap.
  2. Using JOSM load each trail area one at a time and make an OSM XML file with just the data you want outlined:
    1. Select the ways which comprise the trail you want shown.
    2. Create a new data layer (Command-N).
    3. Make the original data layer active.
    4. Copy the selected data from the first layer to your new layer with EditMerge Selection (Shift-Command-M).
    5. Hide the original data layer.
    6. Review the new layer to be sure it has everything you want.
    7. Select all nodes and ways (Command-A) and remove all tags to make later processing easier.
    8. Look good? Is everything you want in the new layer? Save it to a .osm file and do the next trail.
  3. Once you have an OSM file for each trail, convert them to Adobe Illustrator format using this version of osm2ai.pl.
    1. Get osm2ai.pl working on your computer. I run this on macOS, and it works fine on Linux as well. Since it’s a Perl script there are probably some dependencies; likely resolved by installing a few modules.
    2. Process each OSM file with: osm2ai.pl --input infilename.osm --projection mercator --output outfilename.ai
  4. Open each file in Illustrator, combine them into a larger document, make it look the way you want, etc.
  5. Done!

Comments closed

Archiving Gallery 2 with HTTrack

Along with the static copy of the MediaWiki, I’ve been wanting to make a static, archival copy of the Gallery 2 install that I’ve been using to share photos, for 15+ years, at nuxx.net/gallery. Using HTTrack I was able to do so, after a bit of work, resulting in a copy at the same URL and with images accessed using the same paths, from static files.

The result is that I no longer need to run the aging Gallery 2 software, yet links and embedded images that point to my photo gallery did not break.

In the last few years I’ve both seen the traffic drop off, I haven’t posted many new things there, and it seems like the old Internet of pointing people to a personal photo gallery is nearly dead. I believe that blog posts, such as this, with links to specific photos, are where effort should be put. While there is 18+ years of personal history in digital images in my gallery, it doesn’t get used the same way it was 10 years ago.

On the technical side, the relatively-ancient (circa 2008) Gallery 2 has and the ~90GB of data in it has occasionally been a burden. I had to maintain an old copy of PHP just for this app, and this made updating things a pain. While there is a recent project, Gallery the Revival, which aims to update Gallery to newer versions of PHP, this is based around Gallery 3 and a migration to that brings about its own problems, including breaking static links.

I’m still not sure if I want to keep the gallery online but static as it is now, put the web app back up, completely take it off the internet and host it privately at home, or what… but figuring out how to create an archive has given me options.

What follows are my notes on how I used HTTrack, a package specifically designed to mirror websites, to archive nuxx.net’s Photo Gallery. I encountered a few bumps along the way, so this details each and how it was overcome, resulting in the current static copy. To find each of these I’d start HTTrack, let it run for a while, see if it got any errors, fix them, then try again. Eventually I got it to archive cleanly with zero errors:

Gallery Bug 83873

During initial runs, HTTrack finished after ~96MB (out of ~90GB of images) saved, reporting that it was complete. The main portions of the site looked good, but many sub-albums or original-resolution images were zero-byte HTML files on disk and displayed blank in the browser. This was caused by Gallery bug 83873, triggered by using HTTPS on the site. It seems to be fixed by adding the following line just before line 780 in .../modules/core/classes/GallerySession.class:

GalleryCoreApi::requireOnce('modules/core/classes/GalleryTranslator.class');

This error was found by via the following in Apache’s error log:

AH01071: Got error 'PHP message: PHP Fatal error: Class 'GalleryTranslator' not found in /var/www/vhosts/nuxx.net/gallery/modules/core/classes/GallerySession.class on line 780\n', referer: http://nuxx.net/gallery/

Minimize External Links / Footers

To clean things up further, minimizing external links, and make the static copy of the site as simple as possible, I also removed external links in footer by commenting out the external Gallery links and version from the footer, via .../themes/themename/templates/local/theme.tpl and .../themes/themename/templates/local/error.tpl:

<div id="gsFooter">
{*
{g->logoButton type="validation"}
*{g->logoButton type="gallery2"}
*{g->logoButton type="gallery2-version"}
*{g->logoButton type="donate"}
*}
</div>

Remove Details from EXIF/IPTC Plugin

The EXIF/IPTC Plugin for Gallery is excellent because it shows embedded metadata from the original photo, including things like date/time, camera model, location. This presents as a simple Summary view and a lengthier Details view. Unfortunately, when being indexed by HTTrack, selecting of the Details view — done via JavaScript — returns a server error. This shows up in the HTTrack UI as an increasing error count, and server errors as some pages are queried.

To not have a broken link on every page I modified the plugin to remove the Summary and Details view selector so it’d only display Summary, and used the plugin configuration to ensure that every field I wanted was shown in the summary.

To make this change copy .../modules/exif/templates/blocks/ExifInfo.tpl to .../modules/exif/templates/blocks/local/ExifInfo.tpl (to create a local copy, per the Editing Templates doc). Then edit the local copy and comment out lines 43 through 60 so that only the Summary view is displayed:

{* {if ($exif.mode == 'summary')}
* {g->text text="summary"}
* {else}
* <a href="{g->url arg1="controller=exif.SwitchDetailMode"
* arg2="mode=summary" arg3="return=true"}" onclick="return exifSwitchDetailMode({$exif.blockNum},{$item.id},'summary')">
* {g->text text="summary"}
* </a>
* {/if}
* &nbsp;&nbsp;
* {if ($exif.mode == 'detailed')}
* {g->text text="details"}
* {else}
* <a href="{g->url arg1="controller=exif.SwitchDetailMode"
* arg2="mode=detailed" arg3="return=true"}" onclick="return exifSwitchDetailMode({$exif.blockNum},{$item.id},'detailed')">
* {g->text text="details"}
* </a>
* {/if}
*}

Disable Extra Plugins

Finally, I disabled a bunch of plugins which both wouldn’t be useful in a static copy of the site, and cause a number of interconnected links which would make a mirror of the site overly complicated:

  • Search: Can’t search a static site.
  • Google Map Module: Requires a maps API key, which I don’t want to mess with.
  • New Items: There’s nothing new getting posted to a static site.
  • Slideshow: Not needed.

Fix Missing Files

My custom theme, which was based on matrix, linked to some images in the matrix directory which were no longer present in newer versions of the themes, so HTTrack would get 404 errors on these. I copied these files from my custom theme to the .../themes/matrix/images directory to fix this.

Clear Template / Page Cache

After making changes to templates it’s a good idea to clear all the template caches so all pages are rendering with the above changes. While all these steps may be overkill, I do this by going into Site Admin → Performance and setting Guest Users and Registered Users to No acceleration. I then uncheck Enable template caching and click Save. I then click Clear Saved Pages to clear any cached pages, then re-enable template caching and Full acceleration for Guest Users (which HTTrack will be working as).

PANIC! : Too many URLs : >99999

If your Gallery has a lot of images, HTTrack could quit with the error PANIC! : Too many URLs : >99999. Mine did, so I had to run it with the -#L1000000 argument so that it’ll then be limited to 1,000,000 URLs instead of the default 99,999.

Run HTTrack

After all of this, I ran the httrack binary with the security (bandwidth, etc) limits disabled (--disable-security-limits) and used its wizard mode to set up the mirror. The URL to be archived was https://nuxx.net/gallery/, stored in an appropriately named project directory, with no other settings.

CAUTION: Do not disable security limits if you don’t have good controls around the site you are mirroring and the bandwidth between the two. HTTrack has very sane defaults for rate limiting when mirroring that keep its behavior polite, it’s not wise to override these defaults unless you have good control of the source and destination site.

When httrack begins it shows no progress on screen, so I quit with Ctrl-C, switched to the project directory, and ran httrack --continue to allow the mirror to continue and show status info on the screen (the screenshot above). The argument --continue can be used to restart an interrupted mirror, and --update can be used to freshen up a complete mirror.

Alternately, the following command puts this all together, without the wizard:

httrack https://nuxx.net/gallery/ -W -O "/home/username/websites/nuxx.net Photo Gallery" -%v --disable-security-limits -#L1000000

As HTTrack spiders the site it comes across external links and needs to know what to do with them. Because I didn’t specify an action for external links on the command line, it prompts with the question “A link, [linkurl], is located beyond this mirror scope.”. Since I’m not interested in mirroring any external sites (mostly links to recipes or company websites) I answer * which is “Ignore all further links and do not ask any more questions” (text in httrack.c). (I was unable to figure out how to suppress this via a command line option before getting a complete mirror, although it’s likely possible.)

Running from a Dedicated VM

I ran this mirror task from a Linode VM, located in the same region as the VM hosting nuxx.net. This results in all traffic flowing over the Private network, avoiding bandwidth charge.

Because of the ~90GB of images, I set up a Linode 8GB, which has 160GB of disk, 8GB of RAM, and 4 CPUs. This should provide plenty of space for the mirror, with enough resources to allow the tool to work. This VM costs $40/mo (or $0.06/hr), which I find plenty affordable for getting this project done. The mirror took N days to complete, after which I tar’d it up and copied it a few places before deleting the VM.

By having a separate VM I was able to not worry about any dependencies or package problems and delete it after the work is done. All I needed to do on this VM was create a user, put it in the sudoers file, install screen (sudo apt-get install screen) and httrack (sudo apt-get install httrack), and get things running.

Wrapping It All Up

After the mirror was complete I replaced my .../gallery directory with the .../gallery directory from the HTTrack output directory and all was good.

Comments closed