Category: computers

Down the OpenStreetMap Rabbit Hole

Published May 25, 2012

I’ve been interested in making mountain bike trail maps for a while now, resulting in my drawing the official River Bends and Addison Oaks maps. Stony Creek Metropark has some great trails as well, but it’s MTB trail map is unfortunately lacking. It is missing some segments used during races, has some one-way markings that don’t match what’s actually signed on the trail, and isn’t geographically accurate enough to be used for trail planning.

My original intention was to record all of the trail segments at Stony Creek myself using a GPS then draw a map similar to the ones for Addison Oaks and River Bends, but after looking on OpenStreetMap (OSM) I found that most of the trails at Stony Creek were already mapped. Thus if I am willing to license my map in an appropriately open manner I can use this data, saving myself a bunch of time.

Since a few small connectors that I wished to include were missing I headed out to Stony Creek this evening and recorded GPS data for only the missing segments. Most of these were connectors used primarily during races, but as they are passable throughout most of the year I wanted to include them. I then brought the GPX into JOSM, edited the map, and submitted the changes. This is a pretty easy task if one is familiar with basic CAD tools.

While the changes aren’t yet reflected in the main online map, the changes have been submitted and it does show when new data is manually downloaded, so I imagine it’s only a matter of time before it’s visible to the public. I will then use this OSM to make my MTB-specific map, complete with markers for things such as log piles, rock gardens, etc.

I suspect that this ease of editing is going to change my mapping workflow in the future. There’s a very good chance that any new maps which I do will first go into OSM, then this data will be pulled out to generate the actual map. Barring any license constraints, of course.

Looking further it appears that River Bends’ trails aren’t yet on OSM. Looks like I’ve got some work to do there as well…

1 Comment

Moving CiviCRM on Joomla to WordPress

Published May 11, 2012

With recent work done on the MMBA website I’ve had to move the CiviCRM instance from an older copy hosted on Joomla to a new one hosted on WordPress. The general suggestion for doing this is to export and import the data, but this is both a bit frustrating to do (because of a need to specify which columns contain which fields) and PHP kept running out of memory when exporting, even after I moved it up to 256MB per process.

This copy of CiviCRM wasn’t really used, but it did have a lengthy set of contacts that I could not afford to lose. Thus, I looked into what was necessary to move the tables themselves instead of just exporting/importing the data.

I ended up using these steps to get the contact data moved over and it seemed to work well for me. This may not be the best way to do it, and it may introduce some problems down the line, but this did work for me and seems fine thus far, so I wanted to document the steps here:

1) Install WordPress and the CiviCRM plugin with the new installation of CiviCRM going to a different database or table prefix than the current.

2) Ensure that the WordPress CiviCRM plugin works, only going as far as the first screen which prompts to set up an organization name, email addresses, etc.

3) Using phpMyAdmin export all CiviCRM tables to a .sql file except for:

civicrm_acl_cache
civicrm_acl_contact_cache
civicrm_domain
civicrm_cache
civicrm_group_contact_cache

With the option: Add DROP TABLE / VIEW / PROCEDURE / FUNCTION / EVENT statement.

4) To avoid issues with Foreign Key Constraints add this line to the top of the .sql file:

SET foreign_key_checks = 0;

And this to the bottom:

SET foreign_key_checks = 1;

5) If the new set of tables has a different prefix than the old, search/replace in the .sql to change each table name to the prefix used by the WordPress-based CiviCRM instance.

6) Import the .sql into the WordPress-based CiviCRM database using the MySQL command line utility:

mysql -u username -p databasename < civicrm_export.sql

7) Enter the CiviCRM plugin in WordPress and confirm that all the contacts have been moved over.

These steps were performed on the following configuration:

- Apache/2.2.22 (FreeBSD) mod_ssl/2.2.22 OpenSSL/0.9.8q mod_fastcgi/2.4.6
- MySQL 5.5.22 (from FreeBSD Ports)
- WordPress 3.3.2 w/ CiviCRM 4.1.0
- Joomla 1.5.24 w/ CiviCRM 3.4.7

2 Comments

Now With IPv6 and SSL (but no SPDY)

Published May 8, 2012

Since moving to a new server running Apache for hosting websites I’ve been able to make quite a few changes that just weren’t quite possible with lighttpd. One major change is the use of IPv6. Ever since the move I’ve set up all hosted sites with AAAA records and made them available via IPv6, and it’s been working great. While I don’t see much traffic on this yet, I’m happy to know that the sites are all able to be reached this way. For reference:

;; ANSWER SECTION:
nuxx.net. 300 IN AAAA 2607:f4b8:5:2:e611:5bff:feac:3a60

I’ve also been wanting to move my personal site, nuxx.net, to be HTTPS-only. While not particularly important for most public content, this does protect session cookies for various admin functions that I may host under this domain, along with keeping any private content from prying eyes. Thanks to an SSL certificate from the new-to-me registrar gandi.net it was pretty easy to get this set up. I was then able to take the old non-HTTPS vhost and permanently redirect it over to the secure one, and all content simply moved over to the secure side:

<VirtualHost *:80>
    ServerName nuxx.net
    ServerAlias www.nuxx.net
    ServerAdmin c0nsumer@nuxx.net
    Redirect permanent / https://nuxx.net/
</VirtualHost>

The browser test screenshot seen above from Qualys SSL Labs SSL Server Test (live results here) shows that HTTPS is pretty well set up on nuxx.net. I’m sure it could be better, but for now I think it’s pretty good.

Thanks to the magic of this bit of SQL I was able to update all http://nuxx.net links in nuxx.net blog posts to https://nuxx.net, allowing most pages to validate as fully secure. I’ll have to do something similar on the other apps to bring them up to snuff:

UPDATE wp_posts SET post_content = REPLACE(post_content, 'http://nuxx.net', 'https://nuxx.net');

Amusingly, the use of HTTPS seems to dramatically cut down on automated SPAM. Perhaps some of the link spamming tools don’t want to deal with the overhead (or cert validation?) of HTTPS? I’d figure they could just ignore them, though…

Anyway, with all of this in place, I got to thinking about SPDY, Google’s shiny new protocol. An Apache module for implementing it, mod-spdy has been released and there is a FreeBSD Port for it, so I decided to give it a go. While the installation worked and Chrome showed it working (as seen via chrome://net-internals/#spdy). Unfortunately, I ran into a couple bugs. Most notable was that a number of httpd processes failed, and I ran into documented issues #36 and #39 which left me with an odd taste for this module. I’ve since disabled it, but once a new version is released I’ll probably try it again.

Leave a Comment

Timeouts with PHP as FastCGI, phpBB + reCAPTCHA + DNSBL, Apache 2.2, mod_fcgi, and FastCgiExternalServer

Published April 16, 2012

Sunday evening after migrating the MMBA Forum to a new webserver I received email from a user claiming that they were unable to sign up for the forum, receiving an 500 Internal Server Error some time after clicking submit. The problem ended up being the signup page taking longer than expected to run and timing out and was resolved by increasing the timeout by adding -idle-timeout 60 to the FastCgiExternalServer line in the vhost’s config.

More specifically, I’d just moved from an older server running lighttpd to a new one using the venerable Apache HTTP Server v2.2. Both setups had per-vhost FastCGI setups pointing to PHP instances running as the user who owned the vhost, which helps ensure that compromised PHP apps affect only files/sites owned by that the user^†.

For example, lighttpd would be set up something like this:

fastcgi.server = ( ".php" =>
  ( "socket" => "/var/run/php-fastcgi/username/username-php-fastcgi.sock",
    "check-local" => "disable",
    "broken-scriptfilename" => "enable"
  )
)

Apache uses something like this:

FastCgiExternalServer /var/run/php-fastcgi/vhosts/example.com -socket /var/run/php-fastcgi/users/username/username-php-fastcgi.sock
AddHandler php-fastcgi .php
Action php-fastcgi /php-fastcgi
Alias /php-fastcgi /var/run/php-fastcgi/vhosts/example.com

During the forum signup, to help cut down on the number of spammy accounts created, there are both reCAPTCHA and DNS Blacklist checks that occur before the account creation actually happens. These were taking longer than the default 30 second timeout, causing the FastCGI interface to time out and close the connection, resulting in log entries such as this:

[Sun Apr 15 20:00:09 2012] [error] [client 192.168.0.2] FastCGI: comm with server "/var/run/php-fastcgi/vhosts/mmba.org" aborted: idle timeout (30 sec)

This led me to increase the FastCgiExternalServer timeout in mod_fastcgi by adding -idle-timeout 60, doubling it from its default, as follows:

FastCgiExternalServer /var/run/php-fastcgi/vhosts/example.com -socket /var/run/php-fastcgi/users/username/username-php-fastcgi.sock -idle-timeout 60
AddHandler php-fastcgi .php
Action php-fastcgi /php-fastcgi
Alias /php-fastcgi /var/run/php-fastcgi/vhosts/example.com

The problem then went away.

I’m not exactly sure why this cropped up with the move to Apache, but I suspect that on lighttpd there was a considerably longer default timeout. This can be set in the lighttpd config by setting idle-timeout, but I wasn’t able to easily figure out what the default is. It’s possible I’ll have to further tune this further in the future, but at least I now know why the problem was occurring.

^† Yes, I know this isn’t a perfect solution, but it’s been proven to work when sites are compromised by automatic tools that attempt to change/delete all they can. In each case that I’ve experienced the damage has typically been limited to content in that user’s home directory. This would not be good mitigation against something which attempted privilege escalation once on the box, went after the httpd itself, etc.

1 Comment

MMBA Forum Posts during 2012 Iceman Registration

Published March 1, 2012

The image above (click to embiggen) shows the one hour moving average statistics for posts (replies) and topics (new threads) on the MMBA Forum. This morning registration for Iceman opened on USA Cycling‘s website and as a result of the site collapsing under the load many people visited the forum to complain, commiserate, and generally discuss the situation. Most of the traffic seemed to be in this thread, although I don’t keep per-thread statistics beyond what’s already exposed to the end users.

Leave a Comment

Android Phone as BitLocker Recovery Key

Published February 28, 2012

I recently ran into an issue where I needed my BitLocker recovery key, but found that I’d left it at home. This locked me out of my machine for a few hours and was a bit of a hassle. It got me wondering if maybe I should instead keep said key on my phone instead, as it does a pretty good job serving as a USB mass storage device.

It turns out that this is not such a keen idea, as it doesn’t work out very well. On my Nexus One (and presumably other Android phones), when connected to a computer they prompt the user to switch the microSD storage over to USB Mass Storage. Once selected this takes a few seconds. Whenever the phone is disconnected from the USB host it automatically presents the card back to the phone, and only prompts the user once a host is detected.

With BitLocker, one becomes aware of needing this key when the boot is interrupted and the user is prompted to insert a USB mass storage device with the key present. The computer is then rebooted, finds the key, and continues to boot. With an Android device as the USB mass storage device this reboot resets the phone. I’ve found that it’s typically not possible to get the phone into mass storage mode between POST and OS load, particularly as the machines I’ve tried it on seem to reset USB after POST.

I think I’ll just have to stick with a USB flash drive for this.

Leave a Comment

Shattered Hard Drive Platter

Published February 24, 2012

To collect neodymium magnets for rebuilding the not-currently-great drip tray to kegerator / kegged beer cooler I started disassembling more old hard drives, attempting to salvage the magnets. When disassembling one drive I decided it’d be fun to bend the platter, but I (stupidly) didn’t pay attention to the platter material before doing so. The result? Tiny bits of broken glass and amazingly thin foil spraying all over the basement floor. Whoops.

At least it was easy to clean up.

I had little luck collecting useful magnets from the drives because some were too small, some drives only had one, and many of them shattered as I removed them. Instead I’ll just buy a bunch of tiny ones on eBay or Amazon and epoxy them on instead. It’ll be a lot easier.

1 Comment

iPod Battery Replaced

Published February 24, 2012

Roughly six years after buying my black fifth generation iPod I finally replaced the battery. After years of working well it was starting to fail, providing me with only a few hours of music listening while at work, way down from the original 10-15 that I was getting when it was new.

The replacement battery was part number BIPOD550MV30 from OWC, a very reliable online Apple reseller who has been around for years, and cost just over $20 shipped. While I could have purchased a cheaper battery via eBayI wanted to be sure to get one from a reliable place, so if I had problems with it there wouldn’t be an issue returning it.

The battery is now installed and charging, so hopefully by morning it’ll be full and ready to go.

Leave a Comment

MS12-006 and HTTP 408 Errors

Published February 20, 2012

I recently worked on an issue where a new Microsoft patch for a security bulletin, MS12-006, was correlated to a web-based application occasionally returning errors. Specifically, while using a web-based version of a reporting package off of a version of Sun (Oracle) Java Web Server the browser would occasionally return a Times New Roman-font “Request Timeout” message when users clicked report-generating links. The folks who support the server working on the issue initially had spent time uninstalling recent patches and found that removing KB2585542 (the patch for MS12-006 on Windows 7 clients) from the client machine would stop the issue from occurring. They then thought it might be a Windows client issue, which is where I got involved.

After some investigation it turns out that the problem was actually with the web server not handling split (fragmented) SSL/TLS records properly. The installation of the patch for MS12-006 causes IE to, in some cases, split SSL/TLS records into multiple requests. Since the server couldn’t appropriately handle these requests it thought the client closed the connection before sending a complete request and returned an 408 Request Timeout message.

This was illustrated by the HTTP logs on the server which show incomplete requests being returned 408 messages, similar to what follows. The bold line illustrates a request that correlates with the issue:

IP       | Username | Date                 | Request Header              | Status | Content | Length

---------+----------+----------------------+-----------------------------+--------+---------+--------

10.0.0.2 | - | 17/Feb/2012:10:03:25 | "P" | 408 | | 148

10.0.0.2 | -        | 17/Feb/2012:10:01:13 | "POST /TestApplet HTTP/1.1" | 200    |         | 13056

After the installation of the patch for MS12-006 there is a setting which allows this new feature to specifically be used only when called for (“optin mode”, the default), all the time (“enabled for all”), or never (“disabled for all”). Setting the option to “disabled for all” kept the issue from being reproduced, and when set to “enabled for all” it always occurred. (See the Registry Information section of 2643584 for more information on these options and how to set them.) By this I showed that it was this SSL/TLS feature causing the issue communicating with the server. Since this appeared to be an issue only with this particular server the team supporting it is now looking into potential solutions on that side.

While it may be tempting to simply disable this feature on clients, this would handicap one of the security enhancements brought along with the MS12-006 patch. Since this issue isn’t likely to be widespread (else lots of HTTPS sessions would be failing) I believe that the servers should be fixed to be compatible with the client’s standard behavior. Specifically, I believe the issue to actually be with the server not having an RFC2246-compliant TLS implementation and thus it should be fixed.

Hopefully this information will be useful to others who run into such an issue. It’s a bit opaque at first, since it looks to the users like the server is having a problem, but to server like the client is occasionally failing to send proper HTTP requests. Trying to troubleshoot this with Fiddler didn’t help either, because the HTTPS proxying would recreate the request, apparently no longer splitting the SSL/TLS record, which kept the error from appearing.

Here’s a few links which are helpful in understanding this issue, in order of usefulness:

· MS12-006: Vulnerability in SSL/TLS could allow information disclosure: January 10, 2012
· SSL/TLS Record Fragmentation Support – Microsoft blog article describing the issue in reverse, from a few years ago when SChannel didn’t support SSL/TLS Record Fragmentation.
· Microsoft Security Bulletin MS12-006 – Important, Vulnerability in SSL/TLS Could Allow Information Disclosure (2643584)
· HTTP Status Code 408 Documentation
· RFC2246 – See section 6.2.1.

1 Comment

Burroughs Mainframe Cards

Published February 17, 2012

Years ago I got the chance to help empty out a friend’s neighbor’s basement after they passed away. This basement was stuffed with old computers and electronics, and I acquired a number of interesting items from there, many of which are now displayed around the house. (Core memory, old test equipment, notebooks, etc.)

The photos from this excursion can be found here, if you’re interested.

One of the items I’d acquired was a milk crate full of old cards from a Burroughs mainframe. Each is a bunch of discrete components built into what’s essentially a logical unit, designed to perform some function. These days a small integrated circuit would be used to perform the same (or multiple) functions.

I’ve given many of these items away, as in my excitement I ended up with many more items than needed. Everything from large boxes of valves to old, uncalibrated oscilloscopes have found homes with friends who had a better use for them than I did. A little while back I was contacted by a guy named William Donzelli who was interested in the history of the Burroughs mainframe and the cards from it. I didn’t know much about it and couldn’t answer many questions, so a couple days back I sent off this box containing the cards to him. Hopefully he’ll find them enlightening.

1 Comment