DoS on Amazon EC2 instances running WordPress

I was got out of bed with an emergency call from a client at the weekend because his blog was down. This is the one that is getting a lot of traffic, but CPU was spiking up to 100% quite unnecessarily. Apache was running about 50 process on ‘ps aux’, each using a few % of CPU. It was very ugly.

First thing was the Apache error log showed lots of “Unable to allocate memory for pool” errors. The solution was found here and involved a couple of additions to /etc/php5/apache2/php.ini


I had a look through the logs and found nothing untoward and so put it down to APC throwing a wobbler, and something to look into in the morning. CPU load had settled to normal levels and things looked good.

Thirty minutes later, I was out of bed again with another phone call. This time, the problem was a bit more obvious from the Apache error and access logs. There were a few thousand of these: - - [16/Mar/2012:18:50:52 +0000] "GET /{page}/?replytocom=8325 HTTP/1.1" 200 132607 "http://{domain}/{page}" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)" 

Each one had an incremented value for ‘replytocom’ so it was clearly scripted. And these only became obvious when I had reduced MaxClients to 10 which gave Apache chance to serve every request, rather than just choking. Definitely something I’d do again under the same circumstance, as I missed this the first time.

Previously I’ve only ever used the Amazon Security Groups to provide firewalling. But after a bit of searching, I found these ufw commands that did the trick:

sudo ufw deny from #Block the attacker
sudo ufw allow 80 #Allow everyone else to hit the webserver
sudo ufw allow 1022 #This port is used for SSH
sudo ufw enable #Enable the firewall

It’s not ideal, especially if the attacker is on a dynamic IP and can just reset their modem, and start again. But at that time, it was enough. And got an email too.

Tomorrow I’ll have to look at better fixes. I make use of Fail2Ban but have found it doesn’t cope well with monitoring access logs on high traffic sites in the past. But that was when using a regex, and a filter for DoS might not need that, so it might be worth a try. Otherwise it looks like Apache mod_evasive is the way to go. With either of these, the tricky part will be balancing the filters to avoid banning real users whilst still catching the bad guys.

Leave a Comment

NOTE - You can use these HTML tags and attributes:
<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>