Linux firewall and SSH protection configuration

The Linux kernel’s built-in firewall function (netfilter packet filtering framework) is administered by the Linux firewalld.

General information about the concept of a firewall.

The documentation on this page includes:

  • IP_sets for handling large numbers of subnets. This is useful for special treatment of entire countries or large ISPs, for example, by firewalld.

  • SSH brute force attacks handled by firewalld.

  • SSH failed logins causing blacklisting using sshblack and firewalld.

RHEL7/CentOS7 and Fedora firewalld

A nice introduction is RHEL7: How to get started with Firewalld.

The default firewall service is now firewalld. The dynamic firewall daemon firewalld provides a dynamically managed firewall with support for network “zones” to assign a level of trust to a network and its associated connections and interfaces. See Introduction to firewalld.

A graphical configuration tool is used to configure firewalld:

firewall-config

A command line (CLI) configuration tool:

firewall-cmd

may also used to configure firewalld. To make any new rule permanent, run the command with the --permanent flag. See the manual firewall-cmd for further information.

The firewalld configuration files are in the directory /etc/firewalld/zones/ where XML files contain the firewall rules.

See firewalld log messages in:

/var/log/firewalld

Whitelisting IP addresses with firewalld

It is important to be able to whitelist internal IP addresses. Insert a firewalld direct_rule so that any incoming source IP packet (src) from a specific IP subnet gets accepted, for example:

firewall-cmd --direct --add-rule ipv4 filter INPUT_direct 0 -s 192.168.1.0/24 -j ACCEPT

To make the rule permanent, run the command with the --permanent flag.

To open a specific port NNNN:

firewall-cmd --zone=public --add-port=NNNN/tcp --permanent

Then remember to reload the firewall for changes to take effect:

firewall-cmd --reload

Here the value 0 is the priority of the rule (smaller numbers have higher priority, so 0 is the highest priority). Make sure that other direct rules (which might interfere with the whitelist) have a lower priority.

List the rules by:

firewall-cmd  --permanent --direct --get-all-rules

Iptables IP sets and country blocking

IP_sets are a framework inside the Linux 2.4.x and 2.6.x kernel which can be used efficiently to create firewall rules for large numbers of IP subnets.

Depending on the type, currently an IP set may store IP addresses, (TCP/UDP) port numbers or IP addresses with MAC addresses in a way, which ensures lightning speed when matching an entry against a set. The IP_sets can be administered by the ipset utility.

Install IP_sets by:

yum install ipset                # RHEL/CentOS 6 and 7

Read the man ipset manual page.

Country IP blocks

There are providers of country IP blocks available for download and eventual inclusion into IP_sets block lists:

For example, the IP blocks for China may start with:

1.0.1.0/24
1.0.2.0/23
1.0.8.0/21
...

Country IPv6 blocks may be found at IPdeny or https://www.countryipblocks.net/ipv6_cidr.php.

For discussions see:

Create a country geoblock ipset

We use the IPdeny all countries zone files and create an IP set which we name geoblock. We offer a simple script which automatically creates an ipset geoblock data file. Download this Makefile to a new directory.

Edit the COUNTRYLIST list variable according to your needs, then run:

make

to create the IPSET_DATA file /etc/sysconfig/ipset.

Tor exit node blocking

If you wish to block port-scanning from Tor exit nodes then you may download the dynamically updated torbulkexitlist file.

The torbulkexitlist file contains a list of IP-addresses which may be considered as a “country named Tor” and simply added as another zone file like in the above country blocks, for example tor.zone.

Using ipsets in firewalld on RHEL/CentOS 7.3 and above

According to the RHEL 7.3 Release Notes, firewalld has been upgraded from version 0.3.9 to 0.4.3:

The ipset must be configured directly in firewalld. Do not try to run the Systemd service ipset service together with the firewalld 0.4 firewalld service (as we did previously). See manual pages for firewalld.ipset, firewalld.zone and firewall-cmd.

The procedure for RHEL/CentOS 7.3 is:

  • As a prerequisite create the data file /etc/sysconfig/ipset as shown above.

  • Create a new ipset called geoblock, optionally specifying a larger maximum number of elements in the list:

    firewall-cmd --permanent --new-ipset=geoblock --type=hash:net [ --option=hashsize=65536 --option=maxelem=524288 ]

See firewall-cmd for the --option flags.

  • Add ipset geoblock to the zone named drop

    firewall-cmd --permanent --zone=drop --add-source=ipset:geoblock

  • Now load ipset data:

    firewall-cmd --permanent --ipset=geoblock --add-entries-from-file=/etc/sysconfig/ipset

  • Reload the firewalld service:

    firewall-cmd --reload

See the manual firewall-cmd section IPSet Options for a list of ipset subcommands.

To list all ipset entries:

firewall-cmd --permanent --ipset=geoblock --get-entries

Using ipsets in firewalld on RHEL/CentOS 8

In RHEL/CentOS 8 the ipset setup works similarly to the above description for 7.3 (and above).

There is an important problem, however, in CentOS 8.3:

  • Overlapping network ranges in the IPsets causes firewalld failures.

This is described in Bugzilla_1836571 firewalld startup failure: COMMAND_FAILED: ‘python-nftables’ failed: internal:0:0-0: Error: Could not process rule: File exists (reported on Fedora FC32). See error messages in Comment 12.

The current status (Dec. 2020) is:

  1. firewalld needs to use the “auto-merge” feature of sets to a allow element coalescing.

  2. The nftables “auto-merge” feature was introduced in release 0.8.2.

Conclusion: Do not (accidentally) use overlapping IPsets in CentOS 8.3.

Aggregate CIDR addresses

To circumvent the CentOS 8 Firewalld failures in case of overlapping IP ranges, there exist several tools:

  1. Python tool aggregate6.

    First install prerequisites:

    dnf install gcc platform-python-devel

    Then install by:

    pip3 install aggregate6 --user

    Usage:

    ~/.local/bin/aggregate6 --help

  2. Perl script aggregate-cidr-addresses which takes a list of CIDR address blocks and combine them without overlaps. We have a copy of the aggregate-cidr-addresses.pl file, download it to /usr/local/bin/.

    Install prerequisite Perl modules:

    dnf install perl-Net-IP

    Usage:

    cat file1 file2 ... fileN | aggregate-cidr-addresses.pl

Testing the ipset service

Testing the ipset:

  • List all the sets in ipset:

    ipset save

  • Test whether an IP address (here: 111.222.33.44) is in a given IPset geoblock:

    ipset test geoblock 111.222.33.44

Flush the ipset kernel table for this IPset:

ipset flush geoblock

This may be required if you delete ipset entries - subsequently you should restart the IPset service.

Enabling the ipset service on RHEL/CentOS 7.0, 7.1 and 7.2

RHEL/CentOS 7.0, 7.1 and 7.2 (which use Systemd) do not have a method for starting an ipset service. This has been fixed in later releases.

A workaround exists for enabling the ipset service on RHEL7/CentOS7 using scripts from Fedora:

See below for how to start the ipset service.

Using ipset with firewalld (RHEL7/CentOS7 and Fedora)

Insert an firewalld direct_rule so that any incoming source IP packet (src) gets matched against the set of IP addresses in geoblock:

firewall-cmd --direct --add-rule ipv4 filter INPUT_direct 1 -m set --match-set geoblock src -j DROP

Here the value 1 is the priority of the rule (smaller numbers have higher priority, so 0 is the highest priority).

Verify the new rule:

firewall-cmd --direct --get-all-rules | grep geoblock

If testing is OK, you can make this rule permanent:

firewall-cmd --permanent --direct --add-rule ipv4 filter INPUT_direct 1  -m set --match-set geoblock src -j DROP

The file /etc/firewalld/direct.xml will contain this direct_rule.

Blocking SSH brute force attacks with firewalld (RHEL7/CentOS7 and Fedora)

See these pages:

These commands may be tried:

firewall-cmd --direct --add-rule ipv4 filter INPUT_direct 2 -p tcp --dport 22 -m state --state NEW -m recent --set
firewall-cmd --direct --add-rule ipv4 filter INPUT_direct 3 -p tcp --dport 22 -m state --state NEW -m recent --update --seconds 30 --hitcount 4 -j REJECT --reject-with tcp-reset

Note: Logging to syslog is missing from this setup. Apparently firewalld is incapable of logging rejected or accepted packets, see:

When tested OK, add the --permanent flag to the above commands.

sshblack – Automatically BLACKLIST SSH attackers

The sshblack script is a real-time security tool for secure shell (ssh). It monitors -nix log files for suspicious activity and reacts appropriately to aggressive attackers by adding them to a “blacklist” created using various firewalling tools – such as iptables – available in most modern versions of Unix and Linux. The blacklist is simply a list of source IP addresses that are prohibited from making ssh connections to the protected host. Once a predetermined amount of time has passed, the offending IP address is removed from the blacklist.

NOTICE: Since modern RHEL (and clones) as well as Fedora use firewalld in stead of iptables, the sshblack version 2.8.1 from the web-site does not work. All iptables commands in the sshblack scripts need to be replaced by similar firewall-cmd commands.

See also this page Further Securing OpenSuSE 11.1 Against SSH Script Attacks.

Installing and configuring sshblack

Unpack the sshblack tar-ball, a local copy is here: sshblackv281.tar.gz. Consult the sshblack_install instructions, a local copy is here: sshblack-install.txt.

Copy the executable files to a standard executable directory:

cp sshblack.pl bl unbl list unlist /usr/local/sbin/

Configure the sshblack.pl script, at a minimum define the sshblack_Whitelist for your local network by tailoring this line:

my($LOCALNET) = '^(?:127\.0\.0\.1|192\.168\.0)';

For security reasons the sshblack CACHE should be in a private directory rather than the world-writable volatile directory /var/tmp, for example:

my($CACHE) = '/var/lib/sshblack/ssh-blacklist-pending';

Note: The same CACHE variable must also be defined in the helper scripts list and unlist.

Create the private directory (RHEL/CentOS conventional location):

mkdir -v -p /var/lib/sshblack

Other configurable parameters include:

my($REASONS) = '(Failed password|Failed none|Invalid user)';
my($AGEOUT) = 600;
my($RELEASEDAYS) = 4;
my($CHECK) = 300;
my($MAXHITS) = 4;
my($DOSBAIL) = 200;
my($CHATTY) = 1;
my($EMAILME) = 1;
my($NOTIFY) = 'root';

Some malformed SSH attacks generate log entries like:

sshd[30179]: Received disconnect from 206.191.151.226: 11: Bye Bye [preauth]

To blacklist such IPs add an additional rule:

my($REASONS) = '(Failed password|Failed none|Invalid user|Bye Bye \[preauth\])';

See also the sshblack_config page for additional advice.

sshblack logfiles

The sshblack logs to this file, so make sure it exists:

touch /var/log/sshblacklisting

It is a good idea to rotate this logfile on a weekly basis, so create the file /etc/logrotate.d/sshblacklisting with the contents:

# Log rotation configuration for SSH blacklisting
/var/log/sshblacklisting {
      missingok
      notifempty
      weekly
}

Configuring the sshblack service

On RHEL (and clones) as well as Fedora Linux you should set up a Systemd startup script (and not run the sshblack.pl command manually).

EL8/EL9/Fedora and sshblack

A Systemd service file sshblack.service must be installed:

cp sshblack.service /etc/systemd/system/
chmod 755 /etc/systemd/system/sshblack.service
systemctl enable sshblack.service

CentOS7/RHEL7 and sshblack

An EL7-specific startup script init-sshblack-el7 must be used for RHEL7/CentOS7/Fedora with Systemd and firewalld. Install a Systemd service file sshblack.service-el7.

Now add the service and create the private sshblack directory:

mkdir /usr/libexec/sshblack
cp init-sshblack-el7 /usr/libexec/sshblack/init-sshblack
cp sshblack.service-el7 /etc/systemd/system/sshblack.service
chmod 755 /usr/libexec/sshblack/init-sshblack /etc/systemd/system/sshblack.service
systemctl enable sshblack.service

Configure a firewalld chain

Create a BLACKLIST chain rule:

firewall-cmd --permanent --direct --add-chain ipv4 filter BLACKLIST

Then make all new connections to port 22 (SSH) jump to the BLACKLIST chain:

firewall-cmd --direct --add-rule ipv4 filter INPUT_direct 7 -p tcp --dport 22 -m state --state NEW  -j BLACKLIST

It is possible to drop packets from specific IP-addresses and subnets using rich rules like:

firewall-cmd --permanent --add-rich-rule="rule family='ipv4' source address='192.168.0.11' drop"

but we don’t use this yet.

Starting the sshblack service

The sshblack daemon must be started:

systemctl start sshblack.service

There are some useful sshblack_notes explaining some additional useful commands (a local copy is here sshblack-notes.html):

  • list – manually adds an IP address to the blacklist and modifies the $CACHE file accordingly

  • unlist – manually remove an IP address from the blacklist and the $CACHE file

  • bl – a manual blacklisting tool (one liner that modifies the iptables configuration only)

  • unbl – a manual UNblacklisting tool (one liner that modifies the iptables configuration only)

  • iptables-setup – a few shell commands to set up the iptables chains if you don’t want to do it manually.

If you want a list of blacklisted IP-addresses, display the BLACKLIST chain:

/usr/bin/firewall-cmd --direct --get-all-rules | grep BLACKLIST

Checkpoint and restart of sshblack

The sshblack.pl script doesn’t have any checkpoint/restart feature, so preservation of BLACKLIST state across restarts must be done manually. See the Checkpoint and Restart discussion.

The script sshblack-save-state should be downloaded to /usr/local/sbin/ and a new crontab rule should be added to run it every 5 minutes:

# Save the firewalld chain BLACKLIST DROP lines for restarting sshblack
*/5 * * * * /usr/local/sbin/sshblack-save-state

This script will create a restart script /var/lib/sshblack/restart.sh which should also be executed at system boot time using a crontab rule:

@reboot /var/lib/sshblack/restart.sh

This command prints commands to recreate the BLACKLIST from the BLACKLIST CACHE in case it is lost:

awk -F, '{if ($3 > 4) printf("firewall-cmd --direct --add-rule ipv4 filter BLACKLIST 0 -s %s -j DROP\n", $1)}' < /var/lib/sshblack/ssh-blacklist-pending

Firewall and NFS

Open up the NFS client’s firewall to all traffic from the specific NFS-server(s). In general this is accomplished by this command:

firewall-cmd --direct --add-rule ipv4 filter INPUT_direct 0 -s <NFS-server-hostname> -j ACCEPT

Summary of RHEL7/CentOS7 and Fedora firewalld settings

Summarizing the direct_rule commands in the above will result in permanent firewalld settings in the file /etc/firewalld/direct.xml:

<?xml version="1.0" encoding="utf-8"?>
<direct>
  <chain table="filter" ipv="ipv4" chain="BLACKLIST"/>
  <rule priority="0" table="filter" ipv="ipv4" chain="INPUT_direct">-s 192.168.1.0/24 -j ACCEPT</rule>
  <rule priority="1" table="filter" ipv="ipv4" chain="INPUT_direct">-m set --match-set geoblock src -j DROP</rule>
  <rule priority="2" table="filter" ipv="ipv4" chain="INPUT_direct">-p tcp --dport 22 -m state --state NEW -m recent --set</rule>
  <rule priority="3" table="filter" ipv="ipv4" chain="INPUT_direct">-p tcp --dport 22 -m state --state NEW -m recent --update --seconds 30 --hitcount 4 -j REJECT --reject-with tcp-reset</rule>
</direct>

Here we have adjusted the priority values so that the most important rules have the highest priority (smaller numbers have higher priority, so 0 is the highest priority).