I’ve been helping a small business with mitigating Distributed Denial of Service (DDoS) attacks to their WordPress sites. Today the attackers came back and tested our new infrastructure on Google Cloud and Cloudflare.
I’m pretty proud to say our team handled the attack quite well, with the attackers failing to take any of our 4 sites down.
Post-attack Results from CloudFlare
Attacks by country
So yep, one of the sites experienced an 80GB attack which is still small compared to the terabyte and petabyte sized attacks against some of the big companies. However, these attacks were still enough to take down the old server hosted with Siteground for days on end. Clearly the Siteground team wasn’t ready nor prepared to deal with DDoS attacks. I’m hosted on Bluehost but this site has never been on the receiving end of an attack, nor would I care too much as it’s not business critical to be up 24/7, but I am curious how well Bluehost would handle it..
Tips and Lessons Learned
Continent blocking is extremely effective, especially when dealing with a purely domestic business. People in Russia and China have no reason to visit your site, especially during an attack, and Cloudflare has great options features to block entire continents easily.
Rate limiting is a fun feature as well, but make sure it’s set to something higher than what normal users would use to browse your site. We were going to implement this type of response with iptables but why use our own CPU when we can use Clouldflare’s?! We did 100/minute which was tested to be totally fine. 20/minute set off some false positives though.
One annoying thing we learned was that Cloudflare will NOT protect only a subdomain, the main domain needs to be on Cloudflare for it to be willing to protect the subdomain as well. This was tricky as some of our clients owned the domain and were just letting us sit some pages on their subdomains. These pages couldn’t be protected without us registering another domain and using a redirection host to direct the traffic (using DNS redirects resulted in https/privacy errors for some reason).
Backups are key..
If you are running WordPress, there are plenty of great plugins to export your site occasionally in case you need to migrate on short notice. For us, getting the files out took forever because we needed customer service at SiteGround to block the active attacks in order for us to get into the site to retrieve the files. Even the backend Cpanel-like access was completely offline during the attack. My favorite plugin is All-in-one-WP-Migration, but they do charge like $60 for a license if your site goes over 500mb to migrate..
Google Cloud DDoS Defenses
Anyway, once we migrated over to Google cloud we were able to leverage a couple additional things like load balancers and the built-in Google Cloud armor that comes with the load balancers to even further mitigate the attacks. We added a few custom rules as shown below, but upon testing with a VPN it didn’t actually block any foreign traffic like that origin.region_code should do, we’ll see if it has any effect during the next DDoS attack.. We also had to remove some of the canary and other triggers since they were extremely inaccurate and actually stopped our site from working.
If anyone has any ideas on how to better implement the defense I’m all ears. Or if anyone has any ideas of tracking the perps down, I’m planning on looking through the logs and seeing which IPs actually made it through to the servers and seeing if there are any distinguishable patterns in the data. Will be fun! Till next time..