You all know those shiny download tools which open loads of connections on a file to try and get it quicker? Those are crappy and put lots of unneeded load on servers. Here’s an example:
I host this simple page for a friend of mine in the UK for when he needs US-based hosting or some place high speed to distribute files from. As part of this he hosts the animations found on this page, which overall aren’t very big. However, someone in Thailand (125.24.191.195) is deciding to get them as quickly as possible using some stupid download tool.
What I see is that the workload on the httpd is at ~277, up from it’s typical of 2 or 3. netstat shows lots and lots and lots of connections (currently 276) from that box, all of them established.
The http log currently shows 9291 these:
125.24.191.195 rowla.dyndns.org - [11/Jun/2008:16:58:34 -0400] "GET /justin/img/piston_std4.mpg HTTP/1.1" 200 32768 "http://www.wis.co.uk/justin/deltic-engine.html" "Mozilla/4.0 (compatible; MSIE 5.00; Windows 98)"
Load on the box itself is .23, which is tolerable, so I’ll probably let this continue. If it’s still going at midnight I’ll take some action, but for now it’s just a bit of irritation. Yes, I know I could limit connections on a per-IP basis, but I prefer not to do this unless it’s actually a problem. If I do need to block that IP, I’ll probably just fail to return anything on that vhost to that netblock. Hopefully they’ll finish getting their file sooner than that.
If you’d like to see it, here’s the current netstat: netstat_11jun2008_1.txt
Here’s a capture of a minute or so of 45 seconds of traffic with that address. Note that each GET results in a whole conversation of only 10k or so: 11jun2008_weird_1.cap.gz