nuxx.net

From nuxx.net
Jump to: navigation, search
rowla.nuxx.net sitting on my workbench.

About

nuxx.net is my private server colocated at Waveform Technology in Southeast Michigan. It originally started as a personal server just to host a photo gallery and basic website, but has expanded to provide a number of services including web hosting, offsite backup, and mail services for quite a few people. Photos of the current server hosting nuxx.net, rowla.nuxx.net are available here.


Hardware

The hardware used in the current incarnation of nuxx.net, rowla.nuxx.net, has come from many different places; some salvaged, some donated, some purchased new. Here is an overview of the major components currently used:

Enclosure / Case: In Win J508T
Power Supply: FSP Group (Fortron-Source) FSP550-GLC
Motherboard: SuperMicro P3TDLR
RAM: 4x ValueRAM KVR133X72RC3/512 (512MB ECC)
CPUs: 2x Intel Pentium III at 1.0 GHz
Disk Controller: 3ware 8006-2LP
Hard Disks: 2x Seagate ST3320620AS (320GB SATA), 2x IBM / Hitachi IC35L120AVVA07-0 (120GB PATA)
Network Interface: Compaq NC7770 (Broadcom 5701-based)


Operating System

nuxx.net currently runs FreeBSD 6.1-RELEASE with a pared down kernel, rebuilt world (targetted to Pentium III's), and all security patches applied.


Remote Management and Backups

All of nuxx.net is remotely managed via web-based tools (such as phpMyAdmin) and directly from a terminal (via ssh). Various web-based reporting tools such as bandwidthd, Webalizer, and mailgraph are used to provide at-a-glance statistics, should they be needed.

Building on the back of FreeBSD's periodic(8) tool, automatically mailed reports also include health status of attached hard disks, as obtained by smartmontools. smartd(8) itself is used to look at each disk once daily and send email if errors are noticed.

Each night all maildirs, database backups, website content, and other important data is backed up to a local set of disks, and then rsync'd offsite, tunneled via ssh. By using rsync it is possible to ensure that a backup of both the system configuration and the large quantity of data housed on nuxx.net (~86GB as of 16-Jan-2007) is made using minimal bandwidth. A nightly backup typically causes around 500MB of traffic. With the monthly 1000GB allocation from the colocation provider nightly backups typically only account for around 1.5% of allowable monthly traffic. I find this overhead to be acceptable.


Primary Applications

From a remote perspecitve, most individuals visiting nuxx.net and its hosted sites only see a few select web-based applications which are hosted under different users' sites. The most common applications to see running on those sites are Gallery, MediaWiki, and WordPress. Both Gallery and MediaWiki installations have been customized to some degree to work according to personal requirements. SquirrelMail is installed to allow IMAP users to check email remotely.

Behind the scenes, nuxx.net runs lighttpd, MySQL, and PHP supporting most of the web applications. More information on the lighttpd setup used on nuxx.net may be found here.

Postfix, SpamAssassin, Courier-IMAP, maildrop, and a rather modified installation of vMail.Admin handle email. Previously qmail, vpopmail, and qmailadmin were used for email, but upon the latest upgrade I decided to try a different MTA. Thus far I've been quite happy with Postfix, as it seems to be a bit easier to tie in with other applications. Additionally, storing its tables in a database makes flexible web-based management of users easier than it seemed to be with qmail and vpopmail.

Colocation

rowla.nuxx.net is currently colocated at Waveform Technology's Troy, Michigan colocation facility. At US$50 per month for colocation of a 1U or mid-tower server Waveform provides very cheap colocation at a local facility and with very nice connections. One must make an appointment to obtain physical access to their server, but as things are very reliable, this has only been necessary four times since I first started hosting my server there. Once to replace the machine, once to deal with a failed disk, once to remove the machine after the power supply was being questionable, and once to return the server two and a half months later.

I had tried moving my sites to DreamHost after the power supply failure, but their setup worked very, very poorly for me and I ended up moving back to Waveform. I've been very, very happy with Waveform since I placed my first colocated server, bornslippy.nuxx.net there in early 2004 and would easily recommend them to anyone in Southeast Michigan who needs a local, low-cost colocation facility.


Advertising

As you have probably noticed, both the main part of nuxx.net and the photo gallery have banner ads placed at the left side and top of each page. While I am opposed to obtrusive advertising (popups, popunders, flashy banners, etc) I have found ads from Google's Adsense-served text ads to be fairly useful. After all, that's how Jeff initially found out about Waveform.

After realizing the popularity of the Honda Music Link iPod Adapter Review article after it was posted in early 2006, I decided to try out AdSense ads. Finding that I could easily insert inobtrusive, similarly colored ads into what had been whitespace, I figured I would give it a try. After hosting the ads for a few months I found that the resulting payments contribute significantly to offsetting the colocation fees, so I've decided to keep them. I personally do not use an adblocker and I don't find them obtrusive, so hopefully others don't either.


bandwidthd graph of traffic from nuxx.net while being Dugg.

Digg Resistance

On the evening of 03-Mar-2007 I took some rather interesting photos of activated Cyalume lightstick compound swirling around in a toilet. After these photos were linked from both the Make Blog and Digg on 04-Mar-2007 traffic on the site shot up tremendously. Over the course of two days following the link becoming popular on Digg, 30GB of traffic was served from nuxx.net alone. Almost all of this traffic was served from either MySQL-backed Gallery or MediaWiki pages themselves, or direct link to image URLs such as this one, which are PHP-based redirects pulling and serving images from a non-publicly accessible directory. This means that every single page load and image view required a fair bit of action on the server side, not just serving up a static image.

So, how did the server fare? Very well. For a period of time the server was sitting with a load bounding between 5 and 10, but after adding one more PHP FastCGI process to the pool available to nuxx.net itself, load wasn't observed breaking 8. Pages were a little slow to load, but no errors were returned.

During the time the high volume of traffic from Digg started I was connected to the server actively communicating via IRC IRC using irssi via ssh. Despite the huge amount of requests for images coming in I didn't notice for a few hours, as neither email nor my ssh session appeared to slow. Nor were any complaints received from owners of the other domains I host. I only noticed the traffic after visiting Digg's main page, seeing the linked story, and looking at the bandwidthd and checking the current load.

In addition to using lighttpd with PHP as a FastCGI (a known-fast combination for PHP websites), I had set Gallery to Maximum Acceleration for guest users, with a timeout of one day. Gallery then caches database queries so that they don't have to be made one each page load. This helps tremendously, and checking the MySQL process list periodically during peak traffic would regularly show only a single query or so accessing one of the tables which handles nuxx.net's gallery, and these queries may have been from another user (or bot) browsing some unrelated, uncached part of the gallery. Such caching has a downside of not providing updates to dynamic data (image blocks, etc) for non-logged-in users more often than once a day, but as I use very few dynamic modules (for simplicity sake and to help with performance) this isn't an issue for me.

Of course, without sufficient bandwidth the server would have simply choked when sending data. Thankfully, the excellent colocation provider I use handled all the traffic without a glitch. I'm really, really happy with Waveform Technology as an ISP / colocation provider.


Hosted Sites

Here is a partial list of the sites hosted on nuxx.net. Mostly personal sites, there is quite a mix of content there: