I must be doing something right if they’re attacking me:
…the two graphs are not quite to the same scale, but the story is plain enough:
I came under some sort of botnet attack from March 24th which lasted three weeks, and it’s kudos to my provider that I didn’t notice any downtime or indeed anything at all for a couple of days, until I wondered why my logfiles were getting quite so big, so fast.
Soon after it started I mailed my junk-list/league-of-friends thusly:
My blog/website has been under apparent attack by a botnet attempting a DDoS for some time now; I can’t see any point to what they are doing other than trying to exhaust my upstream bandwidth, which is by my contract “unlimited” and so that won’t work.All the clients claim to be:
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322) …and have been iterating the following trivial retreival:
“GET /users/alecm/ HTTP/1.0” …with no referrer-spam or anything to their benefit; I get two or three fetches per second; this has been going on since midnight Eastern Time on Thursday 23rd of March. This also corresponds with a recent fix of an error in my DNS records, as-per my service ticket record; so possibly this problem has been going on for longer, but was not spotted due to DNS misconfiguration.
I noticed the attack in the middle of last week, and although it’s not a serious threat I decided to try and mitigate it; I replumbed my web filesystem so that attempts to retreive /users/alecm failed with 404, but attempts to retreive its contents continued to succeed.
That the attack represents a conscious action on someone’s part was proven in that over this past weekend the fetches have gradually migrated to being retreivals of:
“GET / HTTP/1.0” In the morning the retreivals started off as the former, by the evening all the retreivals were of the latter; also the traffic volume dropped on that day:
-rw——- 1 alecm alecm 3260089 21 Mar 06:02
-rw——- 1 alecm alecm 2969867 22 Mar 06:02
-rw——- 1 alecm alecm 30941690 24 Mar 06:06 [incl 23/Mar]
-rw——- 1 alecm alecm 25907499 25 Mar 06:02
-rw——- 1 alecm alecm 26994545 26 Mar 07:03
-rw——- 1 alecm alecm 27347266 27 Mar 07:02
-rw——- 1 alecm alecm 27949940 28 Mar 07:02
-rw——- 1 alecm alecm 26414576 29 Mar 07:03
-rw——- 1 alecm alecm 26367467 30 Mar 07:05
-rw——- 1 alecm alecm 24017067 31 Mar 07:03
-rw——- 1 alecm alecm 15456054 1 Apr 07:03 [drop]
-rw——- 1 alecm alecm 20631515 2 Apr 07:02
-rw——- 1 alecm alecm 20321901 3 Apr 06:02
…which probably indicated manual intervention to change the direction of the attack, since the weekend previous showed no sign of tailing-off in traffic due to (eg) people switching off machines.
Some analysis of the logfiles:
# grep the retreival, and do a basic check for “NET CLR 1.1.4322”
$ cat [logfiles] | egrep “GET /(users/alecm/)? HTTP/1.0” | grep 4322 > /tmp/2
$ field 1 < /tmp/2 | countlines | sort -n > /tmp/3
$ wc -l /tmp/3
298485 /tmp/3
$ awk ‘$1>10’ /tmp/3 | wc -l
30462
$ awk ‘$1>25’ /tmp/3 | wc -l
11772
$ tail -5 /tmp/3
424 219.93.153.249
431 84.205.223.252
489 203.152.5.4
1044 203.160.1.50
3247 202.156.6.69
…shows that 298,000 separate IP addresses retreived one of those two pages; that number drops to 30,000 if you discard any site fetching fewer than 10 times, and 11,000 machines fetch 25-or-more times.
The biggest sources are as follows:
hits address name
—- ——- —-
[ed: snipped for brevity]
424 219.93.153.249
431 84.205.223.252 proxy4.satfilm.pl
489 203.152.5.4 proxy04.pacific.net.th
1044 203.160.1.50 localhost
3247 202.156.6.69 202-156-6-69.cache.maxonline.com.sg
…I find the resolution of “203.160.1.50” to “localhost” to be particularly sneaky.
Aside from someone trying to flood my bandwidth, can anyone else theorise what might cause this? I regret not knowing anyone inside NTL otherwise we might be able to use bmly-cache-7.server.ntli.net to work out what’s going on…
– and got lots of useful advice, especially from Jerry, John, Jon, Alan and Alan; with this I added some mitigation into Apache and decided to wait it out. Cutting a long story short, on 23rd April I posted the following:
The botnet attack started on March 21st.It stopped on April 16th.
It consumed approximately 3 million GET requests
22:01:29 buster:full-logs $ wc -l z
2964962 z
Traffic was sourced from more than 540,000 IP addresses: [ed: to me that sounds like a sodding enormous BotNet]
22:02:59 buster:full-logs $ field 1 < z | countlines > x
22:06:03 buster:full-logs $ wc -l x
547942 x
It consumed 876Mb of bandwidth (Thanks to Junkies for hints on Apache config that limited this damage)
22:08:59 buster:full-logs $ field 10 < z | perl -ne ‘{$i += $_} END{print “$i\n”}’
876781316
The 20 heaviest-sourcing IP addresses were:
202.57.146.225 404 202.57.146.225.rev-ip.isp-thailand.com
212.76.37.162 408 nat-mi2.aster.pl
217.98.71.252 443 252.nat.ino.tvknet.pl
212.76.37.138 454 nat-tar.aster.pl
83.247.136.48 489
195.69.209.1 528
81.15.226.6 570 nat-1.elb.vectranet.pl
80.51.159.20 588 w3cache.sistbg.net
80.1.224.11 675 bmly-cache-7.server.ntli.net
220.245.178.140 682 syd-nxg-pr9.tpgi.com.au
213.105.224.16 749 swan-cache-6.server.ntli.net
212.56.130.104 845 ts-02.vol.net.mt
203.162.3.157 890
203.152.5.4 924 proxy04.pacific.net.th
84.205.223.252 969 proxy4.satfilm.pl
164.100.210.150 1007
213.190.195.100 1054 webcache-1.netmadeira.net
219.93.153.249 1227
203.160.1.50 2182 localhost
202.156.6.69 7769 202-156-6-69.cache.maxonline.com.sg
…and I still have no idea why it occured, [ed: paranoid theory elided for blog]
Of the resulting suggestions, I like my mate Chris’s idea best:
Perhaps it’s because you wouldn’t give them the passwords for those LJ accounts they kept asking for… 🙂
Leave a Reply