http://linuxexchange.org/questions/391/detecting-a-compromised-host<p>Okay so you have installed your distribution of choice and done all of the recommended hardening security steps such as encrypted paritions, configured your firewall (iptables/tcpwrappers), installed a file integrity checker/host intrusion detection system (tripwire, samhain, ossec). You also regularly check for rootkits using tools such as (rkhunter, chkrootkit) and audit/harden your machine following (lynsis / bastilles) recommendations. Only running services which are necessary and disable any which are surplus to requirements. Keeping up to date with security patches, maintaining good file system permissions and regularly reading your system logs.</p> <p>Although the above is not a fully comprehensive list, If you have implemented the above chances are your in pretty good shape. However my question is what additional steps do you take to protect/prevent your host/network from becoming compromised and more importantly if it was how quickly would you know about it? </p> <p>For example here is a few things that I personally do on my Linux boxes.</p> <ul> <li>Create an MD5 hash of my currently running iptables check this every 15 minutes if it has changed something has been added/removed.</li> <li>I create separate partitions for various mount points such as (/var,/usr,/tmp,/boot,/home) etc and use mount options such as NOSUID, NOEXEC, READONLY. I then check every 15 minutes and compare the current mount options to ensure none of them have changed, If one of them suddenly writable I want to know about it. </li> <li>Monitor netstat's output for listening services and comparing this against a file which contains a list of known good services/ports which I have approved. If any new services start listening I will be informed and can check whether this is legitimate or a potential backdoor.</li> <li>Perform a daily scan of my local home network using nmap/ndiff to detect any foreign hosts or determine if any other hosts are listening for new services.</li> <li>Implement and check that critical files (/etc/passwd, /etc/shadow, /etc/group ...) are immutable (chattr +i).</li> </ul> <p>The above checks can be performed regularly using a cronjob. Obviously if someone roots the box swiftly some of these checks are going to be useless if the attacker gets an opportunity to disable them. Although if you have more than one Linux box you can configure them to check on each other which adds an additional layer of protection.</p> <p>I am keen to know what the rest of the community does and any funky hurdles they place to trip an attacker up?</p>enTue, 11 May 2010 06:38:13 -0400http://linuxexchange.org/questions/391/detecting-a-compromised-host/457<p>Load monitoring is useful. I've seen attacks come thru outdated installations of Wordpress and start up brute force kernel timing attacks. I noticed this because the cpu on the system was unusually busy. I wouldn't have noticed if I hadn't been graphing the output.</p> <p>Rootkit checking are useful. Running log analysis on website logs is useful. I've seen file upload forms get circumvented such that PHP botnet scripts get uploaded (to an image upload folder).</p> <p>Defense in Depth would also encourage application security assessment, not just OS monitoring. So running an application attack test (XSS, XSI, attacks etc), and application-level password audits are just as important. A wordpress login with a weak password is a super wide open door.</p> <p>(Of course, the most important thing is backing up your system and testing your restore procedure. It might not be an attack that brings your site down, but an over-worked admin, an unexpected fire sprinkler shower, who knows what.)</p>memnoch_proxyTue, 11 May 2010 06:38:13 -0400http://linuxexchange.org/questions/391/detecting-a-compromised-host/457http://linuxexchange.org/questions/391/detecting-a-compromised-host/445<p>I use ossec extensively. event log correlation, active response, file system integrity checking, along with hardening the host similar to how you describe in your post.</p>Aaron 1Tue, 11 May 2010 02:33:50 -0400http://linuxexchange.org/questions/391/detecting-a-compromised-host/445http://linuxexchange.org/questions/391/detecting-a-compromised-host/408<p>I use Monit. In addition to checking if services are running it also can check checksums of binaries, init scripts, etcetera. Even better, it can prevent restarting compromised binaries. If an attacker manages to compromise your binaries, Monit will see it and alert you about it. It can also refuse to restart the service so the compromised binary doesn't run.</p>Sander MarechalSun, 09 May 2010 19:39:11 -0400http://linuxexchange.org/questions/391/detecting-a-compromised-host/408http://linuxexchange.org/questions/391/detecting-a-compromised-host/406<p>You could install an IDS like Snort and look for suspicious traffic to/from your different servers. There are a lot of signatures in place to detect malicious traffic, so this could perhaps give an indication of whether a system has been compromised or not.</p>feinomSun, 09 May 2010 16:57:59 -0400http://linuxexchange.org/questions/391/detecting-a-compromised-host/406http://linuxexchange.org/questions/391/detecting-a-compromised-host/395<p>This will stop any cracker from using your box, except the one who is aware of all these checks and ready to immidiately bypass all of them replacing with own scripts. Of course it will only work good if you have enabled email notifications so you can react fast. Otherwise this will not make any sense. You need to think about actions to do in such cases. Say, <em>ifconfig eth0 down</em>, a proven way to wipe out any cracker while you are sleeping or afk :)</p> <p>Don't forget to regularly check for updates. Patching your kernel with grsecurity is a good way too.</p>Web31337Sat, 08 May 2010 14:07:24 -0400http://linuxexchange.org/questions/391/detecting-a-compromised-host/395http://linuxexchange.org/questions/391/detecting-a-compromised-host/393<p>Your post there is pretty conclusive.</p> <p>Something that we find is that once an attacker has any sort of foothold (via a hosted site or whatever) then they tend to start sending spam.</p> <p>We have scripts in place to monitor the number of emails sent per user per X minutes and the total number in the outbound queue and alert us if either trip the limits.</p>gregularexpressionsSat, 08 May 2010 13:19:09 -0400http://linuxexchange.org/questions/391/detecting-a-compromised-host/393