rez.nuxx.net hull breech
Well, I’m pretty certain I know how the attack was done.
It appears to have been a phpBB exploit targeted at crisishour.com launched from 213.219.122.11.
Within the course of ~30 minutes that IP hit every single vhost I’ve got, and on crisishour.com it actively went and ‘did things’ in phpBB. Looks like via whatever hole in phpBB it deleted the contents of every vhost it could find and tossed an index.html or index.php in the root, as appropriate. I believe it also may have parsed the config files for Apache, because it seems like a pretty intelligent script. It also deleted all files and directories that www:www had access to in those subdirectories.
Fortunately things in the database and Gallery data directory are intact. I just need to move all of that over to Dreamhost one site at a time. This will take a while.
For what it’s worth, that IP is zone-h.org, which appears to be a shady ‘security’ site and the IP is owned by people in Estonia.
So, lessons learned?
1) Separate privileges based on site.
2) Don’t host domains for people who won’t keep their shit reasonably up to date and won’t stay in touch with me.
3) Don’t run phpBB until it’s been secure for a while.
root@rez:/var/data/wwwlogs# grep -Rne “213.219.122.11” *
5thelement.com/2006/07/2006-07-08-access.log:106:213.219.122.11 – – [08/Jul/2006:07:53:01 -0400] “GET / HTTP/1.0” 200 1842 “-” “Wget/1.9.1”
abrazos.net/2006/07/2006-07-08-access.log:92:213.219.122.11 – – [08/Jul/2006:07:57:43 -0400] “GET / HTTP/1.0” 200 1842 “-” “Wget/1.9.1”
atomicsatchel.com/2006/07/2006-07-08-access.log:22:213.219.122.11 – – [08/Jul/2006:07:53:01 -0400] “GET / HTTP/1.0” 200 1842 “-” “Wget/1.9.1”
baycitybootymafia.com/2006/07/2006-07-08-access.log:590:213.219.122.11 – – [08/Jul/2006:07:53:01 -0400] “GET / HTTP/1.0” 200 1842 “-” “Wget/1.9.1”
briansphotos.nuxx.net/2006/07/2006-07-08-access.log:56:213.219.122.11 – – [08/Jul/2006:07:53:01 -0400] “GET / HTTP/1.0” 200 1842 “-” “Wget/1.9.1”
bunjamin.net/2006/07/2006-07-08-access.log:123:213.219.122.11 – – [08/Jul/2006:07:53:01 -0400] “GET / HTTP/1.0” 200 1842 “-” “Wget/1.9.1”
crisishour.com/2006/05/2006-05-16-access.log:384:213.219.122.11 – – [16/May/2006:07:09:31 -0400] “GET /forum HTTP/1.0” 301 240 “-” “Wget/1.9.1”
crisishour.com/2006/05/2006-05-16-access.log:386:213.219.122.11 – – [16/May/2006:07:09:31 -0400] “GET /forum/ HTTP/1.0” 200 37323 “-” “Wget/1.9.1”
crisishour.com/2006/05/2006-05-16-access.log:387:213.219.122.11 – – [16/May/2006:07:09:34 -0400] “GET /robots.txt HTTP/1.0” 404 208 “-” “Wget/1.9.1”
crisishour.com/2006/05/2006-05-16-access.log:388:213.219.122.11 – – [16/May/2006:07:09:34 -0400] “GET /forum/templates/AdInfinitum/images/folder_big.gif HTTP/1.0” 200 73 “http://www.crisishour.com/forum/” “Wget/1.9.1”
crisishour.com/2006/05/2006-05-16-access.log:389:213.219.122.11 – – [16/May/2006:07:09:34 -0400] “GET /forum/templates/AdInfinitum/images/icon_latest_reply.gif HTTP/1.0” 200 169 “http://www.crisishour.com/forum/” “Wget/1.9.1”
crisishour.com/2006/05/2006-05-16-access.log:390:213.219.122.11 – – [16/May/2006:07:09:35 -0400] “GET /forum/templates/AdInfinitum/images/whosonline.gif HTTP/1.0” 200 205 “http://www.crisishour.com/forum/” “Wget/1.9.1”
crisishour.com/2006/05/2006-05-16-access.log:391:213.219.122.11 – – [16/May/2006:07:09:35 -0400] “GET /forum/templates/AdInfinitum/images/folder_new_big.gif HTTP/1.0” 200 414 “http://www.crisishour.com/forum/” “Wget/1.9.1”
crisishour.com/2006/05/2006-05-16-access.log:392:213.219.122.11 – – [16/May/2006:07:09:36 -0400] “GET /forum/templates/AdInfinitum/images/folder_locked_big.gif HTTP/1.0” 200 133 “http://www.crisishour.com/forum/” “Wget/1.9.1”
crisishour.com/2006/05/2006-05-16-errors.log:145:[Tue May 16 07:09:34 2006] [error] [client 213.219.122.11] File does not exist: /var/data/www/crisishour.com/robots.txt
dansphotos.nuxx.net/2006/07/2006-07-08-access.log:3:213.219.122.11 – – [08/Jul/2006:07:53:12 -0400] “GET / HTTP/1.0” 200 1842 “-” “Wget/1.9.1”
default/2006/05/2006-05-16-access.log:50:213.219.122.11 – – [16/May/2006:07:09:38 -0400] “HEAD / HTTP/1.0” 200 – “-” “Sprint (safemode.org)”
default/2006/07/2006-07-08-access.log:9:213.219.122.11 – – [08/Jul/2006:07:53:01 -0400] “HEAD / HTTP/1.0” 200 – “-” “Sprint (safemode.org)”
default/2006/07/2006-07-08-access.log:10:213.219.122.11 – – [08/Jul/2006:07:53:02 -0400] “HEAD / HTTP/1.0” 200 – “-” “Sprint (safemode.org)”
default/2006/07/2006-07-08-access.log:11:213.219.122.11 – – [08/Jul/2006:07:53:02 -0400] “HEAD / HTTP/1.0” 200 – “-” “Sprint (safemode.org)”
default/2006/07/2006-07-08-access.log:12:213.219.122.11 – – [08/Jul/2006:07:53:02 -0400] “HEAD / HTTP/1.0” 200 – “-” “Sprint (safemode.org)”
default/2006/07/2006-07-08-access.log:13:213.219.122.11 – – [08/Jul/2006:07:53:02 -0400] “HEAD / HTTP/1.0” 200 – “-” “Sprint (safemode.org)”
default/2006/07/2006-07-08-access.log:14:213.219.122.11 – – [08/Jul/2006:07:53:12 -0400] “HEAD / HTTP/1.0” 200 – “-” “Sprint (safemode.org)”
default/2006/07/2006-07-08-access.log:15:213.219.122.11 – – [08/Jul/2006:07:53:12 -0400] “HEAD / HTTP/1.0” 200 – “-” “Sprint (safemode.org)”
default/2006/07/2006-07-08-access.log:16:213.219.122.11 – – [08/Jul/2006:07:53:13 -0400] “HEAD / HTTP/1.0” 200 – “-” “Sprint (safemode.org)”
default/2006/07/2006-07-08-access.log:17:213.219.122.11 – – [08/Jul/2006:07:57:44 -0400] “HEAD / HTTP/1.0” 200 – “-” “Sprint (safemode.org)”
default/2006/07/2006-07-08-access.log:18:213.219.122.11 – – [08/Jul/2006:07:57:55 -0400] “HEAD / HTTP/1.0” 200 – “-” “Sprint (safemode.org)”
default/2006/07/2006-07-08-access.log:19:213.219.122.11 – – [08/Jul/2006:08:09:34 -0400] “HEAD / HTTP/1.0” 200 – “-” “Sprint (safemode.org)”
default/2006/07/2006-07-08-access.log:20:213.219.122.11 – – [08/Jul/2006:08:13:08 -0400] “HEAD / HTTP/1.0” 200 – “-” “Sprint (safemode.org)”
default/2006/07/2006-07-08-access.log:21:213.219.122.11 – – [08/Jul/2006:08:13:08 -0400] “HEAD / HTTP/1.0” 200 – “-” “Sprint (safemode.org)”
default/2006/07/2006-07-08-access.log:22:213.219.122.11 – – [08/Jul/2006:08:13:08 -0400] “HEAD / HTTP/1.0” 200 – “-” “Sprint (safemode.org)”
default/2006/07/2006-07-08-access.log:23:213.219.122.11 – – [08/Jul/2006:08:13:09 -0400] “HEAD / HTTP/1.0” 200 – “-” “Sprint (safemode.org)”
default/2006/07/2006-07-08-access.log:24:213.219.122.11 – – [08/Jul/2006:08:13:16 -0400] “HEAD / HTTP/1.0” 200 – “-” “Sprint (safemode.org)”
default/2006/07/2006-07-08-access.log:25:213.219.122.11 – – [08/Jul/2006:08:13:19 -0400] “HEAD / HTTP/1.0” 200 – “-” “Sprint (safemode.org)”
default/2006/07/2006-07-08-access.log:26:213.219.122.11 – – [08/Jul/2006:08:13:19 -0400] “HEAD / HTTP/1.0” 200 – “-” “Sprint (safemode.org)”
default/2006/07/2006-07-08-access.log:27:213.219.122.11 – – [08/Jul/2006:08:13:19 -0400] “HEAD / HTTP/1.0” 200 – “-” “Sprint (safemode.org)”
default/2006/07/2006-07-08-access.log:28:213.219.122.11 – – [08/Jul/2006:08:13:21 -0400] “HEAD / HTTP/1.0” 200 – “-” “Sprint (safemode.org)”
default/2006/07/2006-07-08-access.log:29:213.219.122.11 – – [08/Jul/2006:08:15:57 -0400] “HEAD / HTTP/1.0” 200 – “-” “Sprint (safemode.org)”
default/2006/07/2006-07-08-access.log:30:213.219.122.11 – – [08/Jul/2006:08:21:07 -0400] “HEAD / HTTP/1.0” 200 – “-” “Sprint (safemode.org)”
dianeanddon.us/2006/07/2006-07-08-access.log:10:213.219.122.11 – – [08/Jul/2006:07:57:54 -0400] “GET / HTTP/1.0” 200 1842 “-” “Wget/1.9.1”
dingleberrypie.com/2006/07/2006-07-08-access.log:158:213.219.122.11 – – [08/Jul/2006:07:53:12 -0400] “GET / HTTP/1.0” 200 1842 “-” “Wget/1.9.1”
futurearchive.net/2006/07/2006-07-08-access.log:18:213.219.122.11 – – [08/Jul/2006:07:53:11 -0400] “GET / HTTP/1.0” 200 1842 “-” “Wget/1.9.1”
midlandmassage.com/2006/07/2006-07-08-access.log:6:213.219.122.11 – – [08/Jul/2006:08:13:08 -0400] “GET / HTTP/1.0” 200 2267 “-” “Wget/1.9.1”
nuxx.net/2006/07/2006-07-08-access.log:5445:213.219.122.11 – – [08/Jul/2006:08:09:33 -0400] “GET / HTTP/1.0” 200 1842 “-” “Wget/1.9.1”
nythia.com/2006/07/2006-07-08-access.log:10:213.219.122.11 – – [08/Jul/2006:08:13:08 -0400] “GET / HTTP/1.0” 200 1842 “-” “Wget/1.9.1”
pics.joythemonster.com/2006/07/2006-07-08-access.log:240:213.219.122.11 – – [08/Jul/2006:08:13:19 -0400] “GET / HTTP/1.0” 200 1842 “-” “Wget/1.9.1”
pink-envy.com/2006/07/2006-07-08-access.log:2548:213.219.122.11 – – [08/Jul/2006:08:13:19 -0400] “GET / HTTP/1.0” 200 1842 “-” “Wget/1.9.1”
realdolltimeshare.com/2006/07/2006-07-08-access.log:11:213.219.122.11 – – [08/Jul/2006:08:13:18 -0400] “GET / HTTP/1.0” 200 1842 “-” “Wget/1.9.1”
rewiredbg.com/2006/07/2006-07-08-access.log:292:213.219.122.11 – – [08/Jul/2006:08:13:19 -0400] “GET / HTTP/1.0” 200 1842 “-” “Wget/1.9.1”
sayyestolibraries.org/2006/07/2006-07-08-access.log:7:213.219.122.11 – – [08/Jul/2006:08:13:15 -0400] “GET / HTTP/1.0” 200 1842 “-” “Wget/1.9.1”
sithspawn.net/2006/07/2006-07-08-access.log:74:213.219.122.11 – – [08/Jul/2006:08:21:06 -0400] “GET / HTTP/1.0” 200 1842 “-” “Wget/1.9.1”
thefest.org/2006/07/2006-07-08-access.log:6:213.219.122.11 – – [08/Jul/2006:08:13:08 -0400] “GET / HTTP/1.0” 200 1842 “-” “Wget/1.9.1”
trasenstine.com/2006/07/2006-07-08-access.log:263:213.219.122.11 – – [08/Jul/2006:08:15:56 -0400] “GET / HTTP/1.0” 200 1842 “-” “Wget/1.9.1”
tydeus.nuxx.net/2006/07/2006-07-08-access.log:26:213.219.122.11 – – [08/Jul/2006:08:13:08 -0400] “GET / HTTP/1.0” 200 1842 “-” “Wget/1.9.1”
lame
Lame squared.
Evil script kiddies. At least it wasn’t a targetted attack. It still blows though.
fucking script kiddies need to have their fingers broken.
I was just thinking that earlier. That a fitting punishment would be dislocating or breaking one’s finger, then making them sit in a room alone, awaiting treatment, for a period of time equal to that which it takes me to fix the problem.
I’m just somewhat happy that I know how it happened. Takes that bit of mystery out of it, at least.
:)
Yeah… Looks like it was just a script. I think it’s common to query Google for phpBB installs, poke through them (Sprint, which you see there, is an OS fingerprinter), then exploit them.
I found the box in an ‘odd’ state because it was rebooted, but the web server was asking for the passphrase for the SSL key. This has only happened once before in the past three years – when my colo provider lost power. So, I found it all odd…
Restarting the web server threw an odd error message about some directory not being there. Fixed that, restarted it again, tried to hit my Gallery, 404’d. Hit the root of nuxx.net, random poorly written muslim Turkish pseudo-political page. (I’ll post a mirror of that later. I really wish if people would spread political messages that they’d actually write something intelligent. But I digress…)
Because the box was rebooted I suspect that an exploit of the OS itself was either attempted or successful. The webserver was running as user www, which shouldn’t have had access to do this. Therefore I’m considering the box compromised, and the only data I’m moving off of it is images, DB exports, etc.
I could just rebuild it, but that’d require the whole thing to be down during that time, etc. Instead I’m just moving to a managed provider, and I’ll bring the box home and use it to do scheduled, wholly automated backups of both the DBs and data. And I’m throwing off people who I don’t regularly communicate with and wouldn’t just pick up the phone and call.
So, yeah. I guess the best I can say is that it was contained and acted on quickly… But it still sucks. :\ Ah well.
By the way, taking the hint from you I was digging into zone-h’s history briefly, and they sure do sound like a shady security company. Looks like they (he?) likes to deface sites for fun, then archive them and claim them as tags.
It’s probably good that the webserver dameons wouldn’t come up automatically.