Last week Matt Jonkman announced the formation of the Open Infosec Foundation. This foundation has been grant funded to create a new open source IDS/IPS engine. Together with Will Metcalf and of course Matt himself, I will be working on this. We want this to be a real community effort where there is a role for everyone in the infosec community. Developers, admins, vendors, goverments, research, education, everyone. There is a lot of work ahead, but that should be great fun and very inspiring. So far things are interesting already. The discussion mailinglist is growing rapidly with many ppl from the community and industry. A #oisf IRC channel was created today on freenode. Join us there to participate in discussion about this project!
Archive for the ‘IPS’ Category
Open Infosec Foundation founded!
Saturday, October 18th, 2008Snort_inline and out of order packets
Monday, July 30th, 2007In Snort_inline’s stream4 modifications, one of the changes is that out of order TCP packets are treated differently from unmodified stream4. This can cause some new alerts to appear and some unexpected behaviour. So I’ll try to explain what happens here.
First of all let me explain quickly what out of order packets are. To put it simple, TCP packets are send out by the source host in a specific order but can arrive in a different order at the destination. Packetloss, link saturation, routing issues are among many things that can cause this. A Snort_inline specific issue is that when Snort_inline can’t keep up with the packets it needs to process, it will drop packets which causes packetloss. These packets will then have to be resent by the sending host.
Out of order packets become a problem when dealing with stream reassembly. Stream reassembly basically is putting all data from the packets in the right order to get the original data as it was sent. We can’t do stream reassembly if we don’t have all packets. Unmodified stream4 basically ignores gaps in the stream. Designed for passive listening for traffic, it has to deal with packetloss differently than Snort_inline.
Next, some definitions of this functionality in Snort_inline. Out-of-order packets: The number of packets that we have in queue that are out of order for a stream. This means they have a higher sequence number than the next in-sequence packet we are expecting. Out-of-order bytes: The number of bytes of the combined data of the out-of-order packets in the stream. Sequence number hole: A gap between two packets, that can be closed by one or more missing packets.
To prevent Snort_inline from using to much memory on bad connections or when an attacker sends lots of out of order packets, Snort_inline can enforce limits to protect itself. Snort_inline can even force a stream to be completely in-order by dropping all packets that are out of order. Sadly, this has a bad effect on the performance of the connections, so you can set certain limits that balance between performance and protection.
When Snort_inline hits these limits, it will (optionally) fire alerts that look like this:
(spp_stream4) TCP out-of-order packets limit reached for stream
(spp_stream4) TCP out-of-order bytes limit reached for stream
(spp_stream4) TCP sequence number holes limit reached for stream
You can disable the alerts by adding the following option to the preprocessor stream4 line: disable_ooo_alerts. The limits themselves can be adjusted by using the following options: max_seq_holes 2, max_ooo_pkts 25, max_ooo_bytes 7000. These are the values I currently use on my home gateway. I got the idea of implementing these limits from this paper by Vern Paxson. However, it seems to me that his suggestion that at max one sequence hole per stream (even per host) was a bit optimistic. Maybe DSL has more packetloss than the university links he studied.
By default Snort_inline uses the settings that were chosen a bit randomly, so they may not fit your usage. Like with the wscaling, please let me know in a comment what values you use!
Blocking comment spam using ModSecurity and realtime blacklists
Friday, February 23rd, 2007Spammers are known to use compromised hosts from all over the world to send their messages. Many people are blocking or scoring email spam based on realtime blacklist (rbl), which contain ipaddresses of these known bad hosts. In my experience this works fairly well for email. A while ago I noticed in the ModSecurity documentation for version 2.0 that ModSecurity features an operator called rbl, that can be used to check the ipaddress of a visitor with a rbl. So I decided to see if I could use the realtime blacklists to prevent comment spam on my blog. Turns out this works great! In this post I’ll show how to get it working.
(more…)
Snort_inline in svn updated to 2.6.1.3
Thursday, February 22nd, 2007This week SourceFire published a security advisory for (among others) Snort version 2.6.1.2, on which Snort_inline is based. So I took some time to update Snort_inline. Normally this would have taken Will and me quite some time, but since we switched to using svn those days are gone. I was able to update it in under a hour. I was very happy I blogged about the procedure to follow, since I had already forgotten about it 
Will is preparing a release based on this, which should also build with ClamAV 0.90.
Anyway, svn is up to date, so if you are using Snort_inline and rely on the DCE/RPC preprocessor, please pull the code from svn.
Check it out! 
Snort_inline 2.6.1.2 BETA 1 released!
Tuesday, January 23rd, 2007William Metcalf has finally released the new Snort_inline version we have been working on so hard, the first release of our code against Snort 2.6. The last release was in June 2006.
Of course, we continue to lag behind SourceFire, as they just released 2.7.0 BETA 1, but I have good hope that we will be able to keep up a little bit better the following time!
Anyway, get the release from the SourceForge download section!
Snort_inline patch updated to 2.6.1.2
Wednesday, January 17th, 2007With the recent Snort vulnerabilities we had to make a choice if we would backport the fixes to our Snort_inline 2.6.0.2 patch or that we would upgrade to 2.6.1.2. Upgrading makes most sense since SourceFire improves Snort with every release, but since the upgrade process has been very painful the last couple of releases, we weren’t really looking forward to it.
Earlier I wrote about my testing with Subversion for Snort_inline, and I found out that using Subversion made the upgrade procedure much easier and much less time consuming. So upgrading it was. Generally there were little changes to the Snort_inline patch required.
One thing however, messed up the way the new stream4inline code works. A new option in Snort’s Stream4, which is enabled by default, is session dropping. The way it works is that when a packet is dropped, the session it belongs to is instructed to drop every packet from that session from that time on.
This makes sense in many cases, but not in all. In stream4inline, we have created options to drop out-of-order packets, out-of-window packets, bad reset packets, and more. Generally, in these cases we want just drop those individual packets, not kill off the session.
Especially killing the session on bad reset packets would be making it easier to kill sessions by third parties. One might argue that sessions writing outside of the window can be killed, but when looking at the out-of-order limits, this can’t be done.
The out-of-order limits are enforced not because it is bad traffic, but to prevent resource starvation attacks against Snort_inline’s stream reassebler. Out-of-order packets will have to be put in the right order before processing, taking CPU time. Also, they have to be queue’d so re-order, taking memory.
By setting out-of-order limits, the burden of getting the stream in order is on the sender of the packets. He will have to retransmit the right packets first, before sending more out-of-order packets. In this case, we don’t want InlineDrop() to kill the entire session. To deal with this, we introduced InlineDropPacketOnly(), that just drops the packet.
A official beta should be out RealSoonNow(tm)
Setting up Subversion for Snort_inline
Wednesday, January 17th, 2007A reason for the slow development of Snort_inline is that we still weren’t using a version control system. Being sick of this, I decided to setup a private Subversion server to see how we could best use it. One thing that complicates the use of such a system is the fact that we maintain a patch on top of source code not maintained by ourselves. So the system must be able to deal with upstream sourcecode updates.
In the excellent book Practical Subversion, Garrett Rooney suggests the use of so called vendor branches. In this setup the vanilla sources of the upstream Snort would be in the svn repository as well. I’ve decided to experiment with this, and this is how I found it to work.
There are two branches in the svn:
vendor/
trunk/
In vendor, the vanilla source is imported, with tags to the specific releases. So for Snort you will have:
vendor/current
vendor/2.6.0.2
The trunk is first initialized as a copy of vendor/current, after which the Snort_inline specific code is added to the trunk. All modifications to our Snort_inline patch will be done in trunk/.
Where this approach shines is when there is a new upstream version. The procedure is this:
- checkout vendor/current
- update your working copy to the new version
- commit
- create a new tag for the new version.
For going to 2.6.1.2, this also meant removing a few files. After this, you have:
vendor/current
vendor/2.6.0.2
vendor/2.6.1.2
After this, checkout the trunk, and do a merge of the two vanilla trees (2.6.0.2 and 2.6.1.2) into the trunk. This will update our Snort_inline code with the new ‘vendor’ version. This will create a number of conflicts that will have to be resolved manually (because of our changes in Snort_inline), but resolving this turns out to be a lot simpler and less time comsuming than our old method of just copy-pasting the Snort_inline code into the new Snort release.
Anyway, since Will and I were happy about this approach, we have decided to move to the SourceForge.net SVN server, which now contains a trunk with Snort_inline code, soon to be released as Snort_inline 2.6.1.2 BETA 1. But don’t wait for us, you can also checkout your own copy from:
https://snort-inline.svn.sourceforge.net/svnroot/snort-inline/trunk/
Check it out!
Detecting and blocking Phishing with Snort and ClamAV
Sunday, November 12th, 2006ClamAV is a great Open Source virusscanner that can be used for detecting virusses from Snort or Snort_inline using the ClamAV preprocessor. However, by using the anti-phishing and anti-scam signatures by SaneSecurity, this combination can also be used to detect and block phishing and scam attempts. Here is how to set it up.
I’ve decided to run this on my gateway, which is a slow machine. Because I don’t want all my traffic to slow down to much, I’m not going to run the ClamAV defs, only the anti-phishing ones. The default location of the defs on my Debian Sarge system is /var/lib/clamav, so I’ve created a new directory called ‘/var/lib/clamav-phish’. Next I’ve downloaded the defs from SaneSecurity. After unzipping them and the defs were ready.
Next was setting up the clamav preprocessor. For this I used the config line in my snort config:
preprocessor clamav: ports 80, dbreload-time 3600, dbdir /var/lib/clamav-phish, action-drop, toclientonly
This line says that spp_clamav should look for traffic on port 80 that flows to the client. It should use the signatures in /var/lib/clamav-phish/ and it should drop the traffic if a phishing attempt is detected. It also checks once an hour to see if the defs in the directory have been updated, and reloads them if so.
William Metcalf pointed me to a site where you can test this setup. It’s called MillerSmiles.co.uk and it’s an anti-phishing site, with many examples on their site. Opening an example shows this in my snort_inline log:
11/12-18:44:29.581771 [**] [129:1:1] (spp_clamav) Virus Found: Html.Phishing.Bank.Gen636.Sanesecurity.06051701 [**] {TCP} 209.85.50.12:80 -> 192.168.1.2:34915
The site failed to open, so it works just fine!
Update on Snort_inline 2.6.0.2 development
Friday, November 10th, 2006I have spend the last week trying to find a very annoying bug that caused Snort_inline to go into 100% CPU on certain traffic. It kept working, only my P3 500Mhz home gateway slowed down to between 2kb/s and 25kb/s, while normally it handles the full 325kb/s for my DSL line at around 25% CPU.
Snort comes with a number of performance measurement options. In 2.6 –enable-perfprofiling was introduced. Also, –enable-profile builds Snort for use with gprof. Next to those you can use strace and ltrace with the -c option to see the ammount of time spend in the several functions.
I already knew the problem was related to my new Stream4 code, since running Snort_inline without the ‘stream4inline’ option made the problem go away. So my performance debugging and code reviews were focussed on that code. However, the performance statistics showed no functions that took large ammounts of time in Stream4.
Update on using realtime blacklists with ModSecurity
Thursday, March 1st, 2007A few days ago I posted a blog article about stopping comment spam with ModSecurity using realtime blacklists (rbl). While the approach was working, I noted having problems with rules when I tried to match on POST methods in HTTP requests.
Luckily, ModSecurity creator Ivan Ristic was quick to point out where the problem is. I’m using the Core Ruleset for ModSecurity, and one thing that ruleset does is use the ‘lowercase’ transformation. This converts all text from arguments to lowercase, so my ^POST$ match would never be able to match. So like Ivan suggested, using ^post$ solved this part.
Next Ivan pointed out a weakness in the rules. My rules looked for /blog/wp-comment-post.php, and would be easily evaded by just using /blog//wp-comment-post.php. He suggested using the ‘normalisePath’ transformation. I did this, but I also slightly changed the rules to not look for the /blog/ part at all (maybe this makes normalisePath useless, but I decided to rather be safe than sorry).
The rules I’m using now look like this:
SecRule REQUEST_METHOD “^post$” “log,deny,chain,msg:’LOCAL comment spammer at rbl list.dsbl.org’”
SecRule REQUEST_URI “wp-(comments-post|trackback)\.php$” “chain,t:normalisePath”
SecRule REMOTE_ADDR “@rbl list.dsbl.org”
SecRule REQUEST_METHOD “^post$” “log,deny,chain,msg:’LOCAL comment spammer at rbl bl.spamcop.net’”
SecRule REQUEST_URI “wp-(comments-post|trackback)\.php$” “chain,t:normalisePath”
SecRule REMOTE_ADDR “@rbl bl.spamcop.net”
SecRule REQUEST_METHOD “^post$” “log,deny,chain,msg:’LOCAL comment spammer at rbl sbl-xbl.spamhaus.org’”
SecRule REQUEST_URI “wp-(comments-post|trackback)\.php$” “chain,t:normalisePath”
SecRule REMOTE_ADDR “@rbl sbl-xbl.spamhaus.org”
Thanks a lot Ivan Ristic for your comments!
Tags: comment spam, Ivan Ristic, ModSecurity, rbl
Posted in IPS, ModSecurity, Web | No Comments »