November 03, 2016 Categories: GNU/Linux

The Internet of Things are flooding my website

Most people have probably heard about the Internet of Things (IoT) by now and how these devices are constantly drafted for your friendly neighborhood botnet. My little place on the interweb is currently gaining popularity among my DVR based visitors. You know the type: running a public telnet service, using hard coded accounts and may occasionally be seen sporting an additional backdoor on port 31337.

These bots are posing as Googlebot, but they are effortless to identify due to their use of the old and largely abandoned HTTP 1.0 protocol.

# Googlebot imposter
"GET / HTTP/1.0" 403 145 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

# Legit Googlebot
"GET / HTTP/1.1" 200 7436 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

Since the bots are all using the old HTTP 1.0 protocol, we can save some bandwidth by rejecting them and serving a “403 Forbidden” status code instead of any actual content. On Apache HTTP Servers, we can quite easily add a rule enforcing the HTTP 1.1 protocol. This can be done by either adding the rule to the host configuration, or by using a .htaccess file.

# ONLY ALLOW HTTP/1.1
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{THE_REQUEST} !HTTP/1.1$
RewriteRule .* - [F]
</IfModule>

To verify that the remedy is working according to plan I’ll perform a test with curl specifying the HTTP version. First up, let’s try this with HTTP 1.1.

# HEAD request using HTTP/1.1
curl -I https://blog.paranoidpenguin.net/

# SERVER RESPONSE
HTTP/1.1 200 OK
Date: Thu, 03 Nov 2016 18:04:45 GMT
Server: JHVH-1
X-Frame-Options: SAMEORIGIN
Strict-Transport-Security: max-age=15768000
Last-Modified: Thu, 03 Nov 2016 17:39:35 GMT
Accept-Ranges: bytes
Content-Length: 27866
Vary: Accept-Encoding,Cookie
Cache-Control: max-age=3, must-revalidate
Expires: Thu, 03 Nov 2016 18:04:48 GMT
Content-Security-Policy: default-src 'self';
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
Content-Type: text/html; charset=UTF-8

As expected the server was happy to serve the request. Now, let’s see what happens when the same request is being sent by curl using the HTTP 1.0 version.

# HEAD request using HTTP/1.0
curl -I -0 https://blog.paranoidpenguin.net/

# SERVER RESPONSE
HTTP/1.1 403 Forbidden
Date: Thu, 03 Nov 2016 18:05:08 GMT
Server: JHVH-1
X-Frame-Options: SAMEORIGIN
Strict-Transport-Security: max-age=15768000
Connection: close
Content-Type: text/html; charset=iso-8859-1

As instructed by our rewrite rule, the server rejected the request based on the protocol version. Obviously this won’t save us from a real DDoS attack but at least it’ll reflect those stray bots passing by in the dark.