Press "Enter" to skip to content

Category: computers

The New MMBA Trail Guide

I’ve been working on much of the behind-the-scenes stuff for the Michigan Mountain Biking Association (MMBA) website for a few years now. This started with a group of us moving from a broken phpBB2-based site to a Joomla site and a phpBB3 forum. After a couple of years the Joomla site was replaced with the more manageable and updatable WordPress.

Back when on Joomla there was a decent page that presented a Trail Guide that Rob Ritzenhein had built for Joomla which was a searchable list of mountain biking trails within the state along with a ZIP code-based proximity search. With the move to WordPress I had to drop this searchable trail guide and replaced it with a basic, text-based list that had been generated off of the old trail data. After two mis-starts and incomplete attempts to find a replacement during 2012, I was a bit concerned that I wasn’t going to have much luck finding someone who was able to write such a guide.

Out of the blue in late January 2013 I received email from a guy named Jeff Lau who said (in much more polite words) that our existing guide sucks and that he’d like to have a go at making something better. He was very right, and after he showed a very promising proof of concept I was really impressed, so he and I met up and things got rolling. Fast forward a couple weeks and he had software nearly ready to go. I was able to help out with some graphics stuff (made my first sprite sheet) and get the old trail data adapted to the new format and loaded into the site, and get a bunch of interested volunteers to proof and submit updates to the data. Following a few rounds of testing and small tweaks it was ready to release.

Last night Jeff and I were able to launch the site publicly, and thus far it’s been very well received. The guide is no longer an impenetrable wall of text, but instead a dynamic, flowing, zoomable Google-based map paired with a list of trails and their details which simply feels nice to use. Updates are easy, done by putting data in an OpenOffice.org Base database and running an SQL query which generates the CSVs, then uploading them to the trail guide’s data directory. Everything seen by the user is client-side JavaScript with the actual data stored in simple CSV files, so it all runs in the user’s browser and requires nothing special on the server.

I’m really impressed with the work that Jeff did in writing the Trail Guide software. He said he wanted to do something, did it, and turned it around very quickly, just in time for spring. The weather is just starting to warm here in Michigan, and getting this kind of guide ready before people look for it is key. I believe that this guide is one of the most important publicly accessible Michigan mountain bike trail advocacy tools that has been put forward in a while.

I strongly believe that one of the best ways to ensure trails remain good for and open to biking is to keep people using them. Once they reach a critical mass they become self-maintaining (users remove much of the deadfall themselves), and the chance of them becoming closed to bikes diminishes because they are well-used and known cycling areas. Giving people a guide to help them find trails increases usage by increasing accessibility.

This cost of this guide came in at a cost of about 120 hours of Jeff’s time, and 40-50 of mine. (While he kept detailed records I did not, but I did have a few 6 hour evenings of researching trails details for accuracy and entering them in the database one at a time.)

The official announcement of the new trail guide can be found here on the MMBA website, and the trail guide itself can be found here: http://mmba.org/trail-guide

Leave a Comment

osm2ai.pl w/ JOSM XML Support

When working up some map stuff and trying to follow my own OSM workflow I ran into a quirk where the osm2ai.pl script wouldn’t deal with OSM XML coming out of JOSM.

Thank to help from a few folks on IRC I now have this updated version of osm2ai.pl available and it will successfully parse files from JOSM.

The root cause of this problem is the script parsing XML by hand, but as I didn’t have the time (or knowledge) to fix the problem by using an XML parser, this hack which deals with differing quote types and extra spacing, was put into place. It works, and I was able to get the map that I promised to someone complete.

Leave a Comment

Fixing nuxx.net’s Backups

nuxx.net has a backup system which uses rsync to back up the server (and everything it hosts) to my desktop iMac, deduplicating the files by way of rsync’s --link-dest argument to create hard links (detailed info here). Maintaining a 30 day backup history it only uses the space of one full backup plus 29 days of change, which isn’t much given the utility allowed. It’s been running fine for a few years, but about a month or so ago I began getting errors where the target disk for the backups was full, which seemed a bit odd to me. As a stopgap I was able to clean up some other things and keep backups working, but the space was a lot more than normal.

Today I managed to get this resolved, but it took a bit of digging. Below the fold you can see what I found.

Leave a Comment

How To Disable IPv6 w/ Sendmail on FreeBSD 9.0-RELEASE

Due to some issues with Comcast flagging some email I’ve been sending via IPv6 as Spam I wanted to keep mail from being sent this way. Comcast publishes this document explaining how to keep IPv6 mail from being blocked, but I’ve got some rDNS issues to sort out before I can work through all of those. So, in the mean time I simply wanted to stop

It took a bit to figure out how to disable IPv6 in the base Sendmail, but now that I’ve got it done I figured I’d share. This is in 9.0-RELEASE, but I’m sure it applies to many other recent FreeBSD Versions:

Edit /etc/make.conf to ensure that IPv6 is turned off for Sendmail compiles. Add this line to the file:

SENDMAIL_CFLAGS= -UNETINET6

Rebuild Sendmail as described here in the FreeBSD Handbook:

# cd /usr/src/lib/libsmutil
# make cleandir && make obj && make
# cd /usr/src/lib/libsm
# make cleandir && make obj && make
# cd /usr/src/usr.sbin/sendmail
# make cleandir && make obj && make && make install

Then, go into your Sendmail config directory (/etc/mail), and if you haven’t so before, run make all to build your machine-specific Sendmail config files.

Edit hostname.mc and locate the line that reads DAEMON_OPTIONS(`Name=IPv6, Family=inet6, Modifiers=O') and comment it out by adding a dnl in front of it:

dnl DAEMON_OPTIONS(`Name=IPv6, Family=inet6, Modifiers=O')

Compile the Sendmail config and restart Sendmail:

make install
make restart

And, now you’re done! Look at /var/log/maillog to ensure that mail is no longer being delivered via IPv6.

Leave a Comment

Asus Eee Box 1501 Upgrades

I’ve been wanting to make the Asus Eee Box 1501 PC that I’ve got connected to the television perform a bit better, so having a spare SSD hanging around I decided to do some upgrading. Using the personal version of XXCLONE I was able to easily copy the old 250GB drive to the SSD, and the RAM was easy to upgrade once I purchased the right stuff. I’d initially made the mistake of thinking I had a 1501P model when mine was actually the plain 1501. This meant I’d accidentally acquired DDR3 RAM, only noticing my mistake when it didn’t fit into the slot. After another trip back to Micro Center (and a bunch of waiting)

As a result of these upgrades the machine feels much faster, and I hope that the problems I was having with Netflix will be gone. The issues seemed disk related, as things would slow down tremendously whenever there was a bit of disk activity occurring. Running diagnostic tools showed a correlation between slowness and disk busyness too. Since this could also have been caused by paging (I didn’t look at it in enough detail to say if it was or not) I also increased the RAM.

Total price for this upgrade was $63.57 ($59.99 for the RAM, plus sales tax), and was only this much because DDR2 is becoming rare and thus more expensive. The upgrade was done in anticipation of seriously reshuffling the video (broadcast TV, internet streaming, etc) setup around the house. That’ll all hopefully happen within the next week or so.

(The photo above shows the inside of the Asus Eee Box 1501. The disassembly was very easy, but I was strongly disappointed by the wireless antenna. While it looks detachable and has a knurled base, turning it broke the plastic off and revealed it to be nothing more than a snap-on antenna with coax that runs directly to the network card. While not completely broken the antenna is now a bit loose and floppy. Thankfully the RF connection is still intact.)

Exchanging the RAM involved waiting in a line to initiate the return, attempting to find the replacement RAM (which listed 5 pieces in stock, but couldn’t be found), locating a suitable alternate replacement, negotiating a price match to the product which couldn’t be found, waiting for the return to be processed, then waiting for the replacement RAM to be brought to the front of the store. This took a total of ~55 minutes.

5 Comments

External Intel 320 SSD

A couple weeks back, the Intel 320 SSD in my iMac failed, leaving the machine unbootable and the drive appearing as an 8MB volume. This drive was replaced with a new 256GB Crucial M4 SSD and the Intel drive was returned for warranty replacement.

Having no immediate need for another SSD but not wanting to have this one laying around I decided to put it in a Macally PHR-S250UAB external enclosure and use it as another piece of fast storage on my Mac. While the installation went well I’m not quite happy with the performance. Despite being connected via FireWire 800 (IEEE 1394b-2002) couple of benchmark tools report only around 80 MB/sec read and 65 MB/sec write while the spinning piece of rust inside the computer reports something like 105 MB/sec read and 107 MB/sec write. It’ll be much lower latency than a spinning disk, but I’m not sure it’ll provide improvement over the internal disk, especially for things I was hoping to do with it like running VMs.

Maybe I’ll give it a try on my work laptop for VMs there, or maybe I’ll see if it’d work out well in Danielle’s Macbook Pro.

The photo above shows the external enclosure with an Intel-provided case sticker applied to the top in place of Macally’s branding. Except for the residual curved blue logo pieces at the end that didn’t fall outside of the cutoff area, I think this worked pretty well. Here is a photo of it all being assembled.

Leave a Comment

Down the OpenStreetMap Rabbit Hole

I’ve been interested in making mountain bike trail maps for a while now, resulting in my drawing the official River Bends and Addison Oaks maps. Stony Creek Metropark has some great trails as well, but it’s MTB trail map is unfortunately lacking. It is missing some segments used during races, has some one-way markings that don’t match what’s actually signed on the trail, and isn’t geographically accurate enough to be used for trail planning.

My original intention was to record all of the trail segments at Stony Creek myself using a GPS then draw a map similar to the ones for Addison Oaks and River Bends, but after looking on OpenStreetMap (OSM) I found that most of the trails at Stony Creek were already mapped. Thus if I am willing to license my map in an appropriately open manner I can use this data, saving myself a bunch of time.

Since a few small connectors that I wished to include were missing I headed out to Stony Creek this evening and recorded GPS data for only the missing segments. Most of these were connectors used primarily during races, but as they are passable throughout most of the year I wanted to include them. I then brought the GPX into JOSM, edited the map, and submitted the changes. This is a pretty easy task if one is familiar with basic CAD tools.

While the changes aren’t yet reflected in the main online map, the changes have been submitted and it does show when new data is manually downloaded, so I imagine it’s only a matter of time before it’s visible to the public. I will then use this OSM to make my MTB-specific map, complete with markers for things such as log piles, rock gardens, etc.

I suspect that this ease of editing is going to change my mapping workflow in the future. There’s a very good chance that any new maps which I do will first go into OSM, then this data will be pulled out to generate the actual map. Barring any license constraints, of course.

Looking further it appears that River Bends’ trails aren’t yet on OSM. Looks like I’ve got some work to do there as well…

1 Comment

Moving CiviCRM on Joomla to WordPress

With recent work done on the MMBA website I’ve had to move the CiviCRM instance from an older copy hosted on Joomla to a new one hosted on WordPress. The general suggestion for doing this is to export and import the data, but this is both a bit frustrating to do (because of a need to specify which columns contain which fields) and PHP kept running out of memory when exporting, even after I moved it up to 256MB per process.

This copy of CiviCRM wasn’t really used, but it did have a lengthy set of contacts that I could not afford to lose. Thus, I looked into what was necessary to move the tables themselves instead of just exporting/importing the data.

I ended up using these steps to get the contact data moved over and it seemed to work well for me. This may not be the best way to do it, and it may introduce some problems down the line, but this did work for me and seems fine thus far, so I wanted to document the steps here:

1) Install WordPress and the CiviCRM plugin with the new installation of CiviCRM going to a different database or table prefix than the current.

2) Ensure that the WordPress CiviCRM plugin works, only going as far as the first screen which prompts to set up an organization name, email addresses, etc.

3) Using phpMyAdmin export all CiviCRM tables to a .sql file except for:

civicrm_acl_cache
civicrm_acl_contact_cache
civicrm_domain
civicrm_cache
civicrm_group_contact_cache

With the option: Add DROP TABLE / VIEW / PROCEDURE / FUNCTION / EVENT statement.

4) To avoid issues with Foreign Key Constraints add this line to the top of the .sql file:

SET foreign_key_checks = 0;

And this to the bottom:

SET foreign_key_checks = 1;

5) If the new set of tables has a different prefix than the old, search/replace in the .sql to change each table name to the prefix used by the WordPress-based CiviCRM instance.

6) Import the .sql into the WordPress-based CiviCRM database using the MySQL command line utility:

mysql -u username -p databasename < civicrm_export.sql

7) Enter the CiviCRM plugin in WordPress and confirm that all the contacts have been moved over.

These steps were performed on the following configuration:

- Apache/2.2.22 (FreeBSD) mod_ssl/2.2.22 OpenSSL/0.9.8q mod_fastcgi/2.4.6
- MySQL 5.5.22 (from FreeBSD Ports)
- WordPress 3.3.2 w/ CiviCRM 4.1.0
- Joomla 1.5.24 w/ CiviCRM 3.4.7

2 Comments

Now With IPv6 and SSL (but no SPDY)

Since moving to a new server running Apache for hosting websites I’ve been able to make quite a few changes that just weren’t quite possible with lighttpd. One major change is the use of IPv6. Ever since the move I’ve set up all hosted sites with AAAA records and made them available via IPv6, and it’s been working great. While I don’t see much traffic on this yet, I’m happy to know that the sites are all able to be reached this way. For reference:

;; ANSWER SECTION:
nuxx.net.               300     IN      AAAA    2607:f4b8:5:2:e611:5bff:feac:3a60

I’ve also been wanting to move my personal site, nuxx.net, to be HTTPS-only. While not particularly important for most public content, this does protect session cookies for various admin functions that I may host under this domain, along with keeping any private content from prying eyes. Thanks to an SSL certificate from the new-to-me registrar gandi.net it was pretty easy to get this set up. I was then able to take the old non-HTTPS vhost and permanently redirect it over to the secure one, and all content simply moved over to the secure side:

<VirtualHost *:80>
    ServerName nuxx.net
    ServerAlias www.nuxx.net
    ServerAdmin c0nsumer@nuxx.net
    Redirect permanent / https://nuxx.net/
</VirtualHost>

The browser test screenshot seen above from Qualys SSL Labs SSL Server Test (live results here) shows that HTTPS is pretty well set up on nuxx.net. I’m sure it could be better, but for now I think it’s pretty good.

Thanks to the magic of this bit of SQL I was able to update all http://nuxx.net links in nuxx.net blog posts to https://nuxx.net, allowing most pages to validate as fully secure. I’ll have to do something similar on the other apps to bring them up to snuff:

UPDATE wp_posts SET post_content = REPLACE(post_content, 'http://nuxx.net', 'https://nuxx.net');

Amusingly, the use of HTTPS seems to dramatically cut down on automated SPAM. Perhaps some of the link spamming tools don’t want to deal with the overhead (or cert validation?) of HTTPS? I’d figure they could just ignore them, though…

Anyway, with all of this in place, I got to thinking about SPDY, Google’s shiny new protocol. An Apache module for implementing it, mod-spdy has been released and there is a FreeBSD Port for it, so I decided to give it a go. While the installation worked and Chrome showed it working (as seen via chrome://net-internals/#spdy). Unfortunately, I ran into a couple bugs. Most notable was that a number of httpd processes failed, and I ran into documented issues #36 and #39 which left me with an odd taste for this module. I’ve since disabled it, but once a new version is released I’ll probably try it again.

Leave a Comment