First steps towards securing an Apache box on CentOS 5.4

I’m mainly writing this to document some tentative steps that I’ve taken to secure a CentOS server. The server is used for a LAMP-based application with a really predictable URL. The box gets probed a lot. If you have additional questions or comments, please post a comment.

Step 1: Turn on the host firewall

The idea here is to use the built-in iptables firewall in Linux to reject connections to services that you are not running. To do some basic security configuration on a CentOS server, you can start with this at the command prompt:

system-config-securitylevel-tui

This will bring up a traditional text-based user interface that will help you configure some security settings. Make sure that Security Level is Enabled. This just means that the iptables firewall is turned on. Because I am a lazy sysadmin, I disable SELinux. Someday soon, I promise I will learn the intricacies of SELinux and I will enable it on my LAMP servers.

Now select “Customize” and hit Enter. Here is a screen shot of what my settings look like.

This gives you a basic way to enter some firewall settings. Note that this just controls the iptables-based firewall running on the local machine. We are protected by additional hardware firewalls on the network at the University. On an Apache server, we would normally enable SSH, WWW, Secure WWW, and possibly Mail (SMTP), depending on the particular circumstances. This host needs to send and receive mail and so the SMTP port is turned on. As an example only, I show MySQL enabled so that you can see the syntax. It’s “service”:”protocol”, where service can be either the name of the service or its port number. We would not normally enable remote hosts to connect to MySQL.

Step 2: Install ModSecurity

ModSecurity is a well-known hardening service for the Apache web server. According to its developer, Ivan Ristic, ModSecurity is an “open source web application firewall”. There are lots of resources on how to use ModSecurity and it features prominently in the book, “Hardening Apache“, by Tony Mobily. As an aside, I highly recommend the Mobily book. The notes that follow barely scratch the surface of configuring ModSecurity.

Make sure that you have gcc and friends installed so that you can compile ModSecurity from source: sudo yum groupinstall “Development Tools”

Install various other dependencies. These can be satisfied through RPM packages:

sudo yum install httpd-devel pcre pcre-devel libxml2-devel curl-devel

Now download modsecurity and do the usual configure, make, make install routine:

wget http://www.modsecurity.org/download/modsecurity-apache_2.5.12.tar.gz
tar xvfz modsecurity-apache_2.5.12.tar.gz
cd modsecurity-apache_2.5.12
cd apache2/
./configure –with-apxs=/usr/sbin/apxs
make
sudo /sbin/service httpd stop
sudo make install

ModSecurity requires rules in order to operate. To make getting started with ModSecurity easier, the ModSecurity team provides a core rule set. This is from the README for the core rule set:

Keep in mind that a predefined rule set is only part of the work required to
protect your web site. We strongly urge you to consult Ivan Ristic’s book,
“ModSecurity Handbook” http://store.feistyduck.com/products/modsecurity-handbook
and the ModSecurity Reference Manual – http://www.modsecurity.org/documentation/.
The CRS is heavily commented to allow it to be used as a step-by-step
deployment guide for ModSecurity.

We need to deploy the core rule set. Copy the core rule set to an appropriate directory. Here is the way that I set it up; however, I don’t really think this is the correct place to put these rules:

cd ../../modsecurity-apache_2.5.12/rules
sudo mkdir /etc/httpd/conf.d/modsecurity_crs/
sudo cp -R * /etc/httpd/conf.d/modsecurity_crs/
cd /etc/httpd/conf.d/modsecurity_crs/

Edit the core rule set to add a location for modsecurity data files:

sudo vim /etc/httpd/conf.d/modsecurity_crs/modsecurity_crs_10_config.conf

In the modsecurity_crs_10_config.conf, add this line:

SecDataDir /var/log/modsecurity/data

Save and exit and create the correct path and correct the ownership of the directory:

sudo mkdir -p /var/logs/modsecurity/data
sudo chown apache.apache /var/logs/modsecurity/data/

Activate the various required Apache modules (including mod_unique_id) by adding or uncommenting these lines in /etc/httpd/httpd.conf:

LoadFile /usr/lib/libxml2.so
LoadModule unique_id_module modules/mod_unique_id.so
LoadModule security2_module modules/mod_security2.so

<IfModule security2_module>
Include conf.d/modsecurity_crs/*.conf
Include conf.d/modsecurity_crs/base_rules/*.conf
</IfModule>

Now, you’re ready to restart Apache:

/sbin/service httpd start

Immediately start checking the logs for errors!

tail -f /var/log/httpd/error_log

The core rule set will begin logging everything to the standard Apache error_log.

By the way, a production server should not have a development environment installed. Your goal is to make it as difficult as possible for an intruder to do anything useful on your server (such as compile software), even if they gain elevated privileges. So, now that we’ve compiled modsecurity, we can remove the development environment: yum groupremove "Development Tools". This will then break apache – as it will remove the libxml library, which is required for ModSecurity. Fix the error by doing this:

yum install libxml2-devel

That should solve the problem. Try restarting apache/httpd and other necessary services and verify that nothing is broken.

“Manually” blocking hosts using the firewall

Once your host is up and running, you should begin to receive email from logwatch with reports of suspicious activity. If you are not receiving logwatch reports, you should double-check that you have arranged to have root’s mail forwarded to a real account. By default, root’s mail stays on the local machine.  Edit /etc/aliases and locate these lines:

         # Person who should get root's mail
         #root:		marc

Add a line after #root: above, that sends root’s mail to a real email address or two:

         # Person who should get root's mail
         #root:		marc
         root: admin1@mydomain.com,admin2@mydomain.com

Now, update the aliases file by running the command “newaliases” from a command prompt. This will update the database of aliases and root’s mail will now be delivered to admin1@mydomain.com and admin2@mydomain.com. Every evening you should receive a logwatch report. Read these reports.

Here’s an example of some information from a logwatch report:

 ################### Logwatch 7.3 (03/24/06) ####################
        Processing Initiated: Sat May 29 04:02:26 2010
        Date Range Processed: yesterday
                              ( 2010-May-28 )
                              Period is day.
      Detail Level of Output: 0
              Type of Output: unformatted
           Logfiles for Host: yourhost.com
  ################################################################## 

 --------------------- httpd Begin ------------------------ 

 A total of 7 sites probed the server
    121.88.4.141
    174.121.217.37
    192.167.114.31
    200.165.57.238
    209.44.120.3
    216.244.69.101
    80.233.172.42

 Requests with error response codes
    403 Forbidden
       /: 2 Time(s)
       //index.php?pg=../../../../../../../../../ ... self/environ%00: 1 Time(s)

Logwatch has helpfully identified seven sites that should probably be blocked from accessing our web server. You could now manually edit the iptables rule sets to prevent these machines from connecting to your web server. As usual in the open source world, someone has written a nice little script that manages this for us. As is also common, I’ve customized the script for use on our server.

#!/bin/sh
# Taken from here:
# http://www.cyberciti.biz/tips/howto-block-ipaddress-with-iptables-firewall.html
# NOTE: I'm blocking before sending to RH-Firewall
# So have substituted -I for -A in the input chain.
###########################################################

BLOCKDB=/etc/ip.blocked
COMMENT=^#
IPS=$(grep -v $COMMENT $BLOCKDB)
for i in $IPS
do
 /sbin/iptables -I INPUT -s $i -j DROP
 /sbin/iptables -A OUTPUT -d $i -j DROP
done

/sbin/iptables -L | mail -s "New IPTables on VSRV8" root

Copy this script into your home directory into a file called ipblocker.sh. Now, create a file called /etc/ip.blocked. Paste the IP addresses from the logwatch email into the /etc/ip.blocked file, one address per line. Now run the script: ./ipblocker.sh. It will add any IP address in /etc/ip.blocked to the appropriate iptables rule set and those hosts will now be silently dropped when they attempt to communicate with the server. An email with the new rules is then emailed to root. Note that these rules will be lost if iptables is restarted, so the script will need to be re-executed.

This is an example of a dead-simple, but fairly labor intensive way to block hosts at the local firewall. Now we’ll look at much more sophisticated tools for blocking hosts based on their behavior.

Automating updates to the iptables firewall

The python program fail2ban can be used to take a variety of actions based on logged evidence of attempts to probe or access the server.  The fail2ban script works by analyzing log files. If there have been too many failed attempts to access a particular service, fail2ban then blocks the offending host by adding rules to the local firewall. Fail2ban can be configured to work with ModSecurity. Here’s one way to install and activate fail2ban, based on an excellent quick how-to from Sonora Communications. You can read their article for the full details. I customized their approach slightly and converted it to a shell script. Here’s the script:

#! /bin/sh
#################
# Based on this how-to: http://www.sonoracomm.com/support/18-support/228-fail2ban
#################

MIRROR=hivelocity
PKG=fail2ban-0.8.4.tar.bz2
DIR=$(basename $PKG .tar.bz2)
URL=http://downloads.sourceforge.net/project/fail2ban/fail2ban-stable/${DIR}/${PKG}?use_mirror=${MIRROR}

wget $URL && tar -xjvf $PKG && cd $DIR
python setup.py install
vim /etc/fail2ban/jail.conf
cp files/redhat-initd /etc/init.d/fail2ban
chmod +x /etc/init.d/fail2ban
chown root.root /etc/init.d/fail2ban
/sbin/chkconfig --add fail2ban
/sbin/chkconfig fail2ban on
/sbin/service fail2ban start

As you can see from the script, this downloads the fail2ban package and installs it. After that, it opens an editor for you to edit the master fail2ban configuration file “jail.conf”. Once you’ve edited the file and saved it, the script continues, copying the appropriate RedHat/CentOS init scripts into place. The script then makes sure that fail2ban will be started on reboot and also launches fail2ban. You can read more about configuring fail2ban for specific scenarios at the fail2ban wiki page at this link: http://www.fail2ban.org/wiki/index.php/HOWTOs.

One of the HOWTOs covers how to integrate fail2ban with ModSecurity. We will need to adjust the ModSecurity core ruleset that we installed earlier in order to enable fail2ban and ModSecurity to work together. This article has already gotten to be too long, so I will leave that for another post.

Please share your thoughts on securing CentOS and apache in the comments below.

Reblog this post [with Zemanta]
Advertisements

4 thoughts on “First steps towards securing an Apache box on CentOS 5.4

  1. Hi Ben

    Just a thank you for the post, i’m exploring more of the linux
    community and the benefits of using linux over the windows
    platform.

    Something else now for me to look into.

    James

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s