Table of Contents
Intro
I’ll show you the problem
Okay so you know that when you host a website/have a server opened to internet connections, there will be a lot of bots interacting with your server.
If you own a server/virtual personnal server and that you have a website. Try to check the access.log of your nginx server :
cat /var/log/nginx/access.log
You might find a whole bunch of lines, like people browsing your website, google, openai and others getting the robots.txt etc.
But there is also those people that want to harm you. Those people are not trying to cause harm to you especially, they just have those awful python crawlers that make those HTTP requests over and over. I will give you a quick example right now.
Do this :
grep "404" /var/log/nginx/access.log
You will get a bunch of logs.
Here are some guy trying to find some interesting stuff on my server (I am not going to show all the lines as that only today, in 12 hours, I got 10000 bots requests, without joking):
[SOME_IP_ADDRESS] - - [DATE] "GET /cgi-bin/info.php HTTP/1.1" 404 19 "-" "python-httpx/0.24.1"
[SOME_IP_ADDRESS] - - [DATE] "GET /cgi-bin/phpinfo.php HTTP/1.1" 404 19 "-" "python-httpx/0.24.1"
[SOME_IP_ADDRESS] - - [DATE] "GET /cgi-bin/info.php.save HTTP/1.1" 404 19 "-" "python-httpx/0.24.1"
[SOME_IP_ADDRESS] - - [DATE] "GET /.env HTTP/1.1" 404 19 "-" "python-httpx/0.24.1"
[SOME_IP_ADDRESS] - - [DATE] "GET /.env.local HTTP/1.1" 404 19 "-" "python-httpx/0.24.1"&
[SOME_IP_ADDRESS] - - [DATE] "GET /.git/config HTTP/1.1" 404 19 "-" "Python/3.10 aiohttp/3.13.1"
[SOME_IP_ADDRESS] - - [DATE] "GET /.gitlab-ci.yml HTTP/1.1" 404 19 "-" "Python/3.10 aiohttp/3.13.1"
They try to get the git configuration of the repository .git/config, your CI/CD gitlab script .gitlab-ci.yml, some ENVIRONMENT VARIABLES .env, etc.
Fortunately, I use a static site generator called hugo, so the root of this webpage is located in a folder that only contains html, css and images, I don’t have any php, .env, cgi scripts thing at all. So they only get 404 errors.
But please be careful about who makes GET and POST requests to your server. You can setup your NGINX server to block specifics path (like deny access to yoursite.com/importantfile.txt etc).
The thing
I am tired of these bots requests. All this computing power could be used for something else, but instead they prefer to waste the resources of our servers.
I knew that having a server would be something “risky” in itself, but I didn’t know that, that many bots are pinging you, refreshing the page all the time, sending POST, DELETE and others requests. It’s basically wasted bandwidth.
I even have some dude bruteforcing my email server. He just have those 30 IPs all starting with the same numbers and he keeps trying to connect as ‘common’ users like kevin, andrea, git, root, postmaster etc.
I am tired of this, so let’s block them.
fail2ban
Presentation
fail2fan is a program that tracks the content of your programs logs (NGINX, dovecot, postfix, ssh, counter-strike, apache…) and grep specific types of REGEX patterns. These regex are defined as filters, they “filter” the content of the logs, searching for matching results.
Fail2ban uses your system’s firewall to ban IPs that make logs that are detected by filters, and then decide to ban them for a certain amount of time.
Installation
Install fail2ban with your package manager, you already have it if you used the emailwiz script from Luke Smith.
# On debian based server
apt-get update
apt-get install fail2ban -y
systemctl enable --now fail2ban
systemctl status fail2ban # should say active (running)
Configuration
After installing fail2ban, you should have tons of filters in /etc/fail2ban/filter.d/.
Fail2ban uses “jails”, they define common rules like, which log should be analyzed, which filter to apply, if someone does match the filter, should you ban him immediatly ? For how long ? etc.
A default config is provided in /etc/fail2ban/jail.conf, don’t edit it as your package manager will overwrite the content if fail2ban’s maintainers make changes to that file.
Before doing anything, please read the top of the “/etc/fail2ban/jail.conf” file. Watch some videos, read the ArchWiki, get your feet wet.
My configuration
This is the configuration I use for fail2ban. You can use it and edit it as much as you want.
# My personnal fail2ban jail config :)
#
# It's a really evil config, you should raise the value of
# "maxretry" accross the file and lower bantime
# to not block legitimate users (like you).
#
# For the "filters", see /etc/fail2ban/filters.d/
# (these are files managed by Fail2Ban and your Package Manager)
# Default values
# They can be overriden by each subsections
[DEFAULT]
bantime = 24h
findtime = 6h
bantime.increment = true
maxretry = 2
action = iptables-allports
backend = auto
# ignoreip won't ban specified IP addresses
# Put the local IPs of your network if desired
# For example if they all start with 192.168.0
ignoreip = 192.168.0.0/24
# SSH
# Enable it if you use SSH outside of your local network.
[sshd]
enabled = false
port = ssh
logpath = %(sshd_log)s
maxretry = 3
bantime = 3700
findtime = 3600
# Nginx configuration
# On debian 13 (for now) the nginx filters don't work
# if we don't define "backend = auto" (See [DEFAULT] section)
# They also don't work without specifying the logpath
[nginx-bad-request]
enabled = true
port = http,https
logpath = /var/log/nginx/access.log
filter = nginx-bad-request
[nginx-botsearch]
enabled = true
port = http,https
logpath = /var/log/nginx/access.log
filter = nginx-botsearch
# Only triggers if limit req is enabled in your nginx conf
# See mine at etc/nginx/conf.d/antispam.conf
# And the line in etc/nginx/sites-available/mielota.com
[nginx-limit-req]
enabled = true
port = http,https
log_path = /var/log/nginx/error.log
filter = nginx-limit-req
[nginx-forbidden]
enabled = true
port = http,https
logpath = /var/log/nginx/access.log
/var/log/nginx/error.log
filter = nginx-forbidden
# [nginx-error-common]
# enabled = true
# port = http,https
# logpath = /var/log/nginx/access.log
# /var/log/nginx/error.log
# filter = nginx-error-common
# [nginx-http-auth]
# enabled = true
# port = http,https
# logpath = /var/log/nginx/access.log
# /var/log/nginx/error.log
# filter = nginx-http-auth
# EMAIL SERVER CONFIGURATION
# HARDENED https://github.com/lukesmithxyz/emailwiz
# If you used emailwiz.sh, the script created a file named
# emailwiz.local in etc/fail2ban/jail.d/
# Remove it if you are going to use the configuration below
[postfix-sasl]
enabled = true
maxretry = 0
[sieve]
enabled = true
maxretry = 0
[postfix]
enabled = true
mode = aggressive
maxretry = 0
[dovecot]
enabled = true
mode = aggressive
maxretry = 0
After saving the changes you made to your jail.local, restart fail2ban.
systemctl restart fail2ban
Using fail2ban client
You can use fail2ban-client to do some cool things.
- To list the ip that you banned :
fail2ban-client banned
- To list the ip that you banned per section
fail2ban-client status SECTION
Example :
fail2ban-client status nginx-limit-req
fail2ban-client status dovecot
- Unban an IP (useful if your server banned you)
fail2ban-client set SECTION unbanip IP_ADDRESS
Conclusion
It’s a bit terrifying to see so many people trying to breach the security of your system. I didn’t know at all that my poor server was suffering that much, this article is only here to warn you, check your logs.
There is tons of tutorial on the web about fail2ban, I only covered the basics.
This is not a guide about fail2ban, it’s really just a warning.