Archive for the ‘IPS’ Category

« Older Entries
Newer Entries »

Compiling Suricata 0.8.2 in Ubuntu Lucid 10.04 in IPS (inline) mode

Saturday, May 1st, 2010

Yesterday I wrote about how to compile and install Suricata 0.8.2 as an IDS on Ubuntu Lucid 10.04, today I’ll explain the steps to compile and install it as an IPS. In IPS mode the engine runs in inline mode. This means that it gets it’s packets from netfilter and sets a verdict on them after inspecting them. This way we can drop packets that trigger the rules.

First, make sure you have the “universe” repository enabled. Go to the System menu, Administration, Software Sources. There enable “Community-maintained Open Source Software (universe)”. If you’re not running a gui, edit /etc/apt/sources.list and enable the universe repository there. Don’t forget doing an “apt-get update”.

Install the following packages needed to build Suricata: libpcre3-dev libpcap-dev libyaml-dev zlib1g-dev libnfnetlink-dev libnetfilter-queue-dev libnet1-dev.

apt-get install libpcre3-dev libpcap-dev libyaml-dev zlib1g-dev libnfnetlink-dev libnetfilter-queue-dev libnet1-dev

Download Suricata 0.8.2 here

Extract the suricata-0.8.2.tar.gz file as follows:

tar xzvf suricata-0.8.2.tar.gz

Enter the extracted directory suricata-0.8.2.

Run “./configure –enable-nfqueue”
If “./configure –enable-nfqueue” was succesful, run “make”
If “make” was succesful, run “sudo make install”
Except for Suricata itself, the build process installed “libhtp”. For that to work properly, run “ldconfig”.

Run “suricata -V” and it should report version 0.8.2.

To use Suricata in inline mode, pass -q <queue id> to the command line. Example:

suricata -c /etc/suricata/suricata.yaml -q 0

Tags: , , ,
Posted in IPS, oisf, Suricata, Ubuntu | 1 Comment »

Suricata 0.8.2 released

Monday, April 19th, 2010

Today the OISF development team released 0.8.2 of the Suricata IDS/IPS engine. I feel this is definitely the best release so far. Read the announcement here. In short, stability was improved, memory footprint reduced, performance improved and new features were added.

One of the tools we used to help improve the engine is a fuzzer created by Will Metcalf, our QA lead. In short, the script takes a pcap file, runs it through editcap (part of wireshark) altering a number of random bytes, then feeds the altered pcap file to Suricata. This resulted in many interesting corner cases. Naturally the script makes sure you don’t forget to enable “ulimit -c unlimited” and such :) More on that script can be found on Will’s blog node5.

For the next period we’ll be working on resolving a number of open issues. There are still a number of improvements we need to make to the relation between our app layer decoding modules and our detection engine. Next to this we’re still missing support for a number of rule keywords, such as asn1 and http_headers. We’re also working on getting our CUDA accelaration into a more usable shape. This release improved it slightly, by making it work on x86_64, but it’s still not useful in production environments.

So as usual, enough to do! Meanwhile, we’re looking for feedback on our release!

Tags: , ,
Posted in ids, IPS, oisf, Suricata | 1 Comment »

Suricata 0.8.1 released

Saturday, February 20th, 2010

Yesterday the OISF development team released Suricata 0.8.1. This release is much improved from our December 31st release. It is way more stable, performs better and has more features. Thanks to the now included HTP library we have much better HTTP handling. The stream engine has seen massive improvements. Initial experimental CUDA code has been added. Initial Win32 support has been added. We’ve added number of missing rule keywords. Many bugs were fixed.

Personally I’m very excited about the help we have gotten from the community. Quite a few patches from community members were applied in this release. Thanks everyone!

Next week the OISF team and a number of experts are meeting up in Istanbul. We’ll be working on crunching a number of technical challenges, sharing ideas and we will start our brainstorming on future development. If you have any ideas about where you think IDS/IPS should go, please let us know so we can discuss it and possibly include it in our future plans.

Tags: , , ,
Posted in ids, IPS, oisf, Suricata | No Comments »

OISF IDS/IPS engine prototype intro

Wednesday, January 7th, 2009

For over a year I’ve been working on a prototype implementation of a new IDS/IPS engine for the Open Infosec Foundation. This is not necessarily going to be the engine we’ll be using in OISF, although it’s likely that at least some of the code will be used. Discussions about features for the engine are still ongoing (wiki, list), once that settles down we’ll see whats usable and whats not. In the worst case I still think many parts like hashing functions, pattern matcher implementations, protocol decoders, etc can be used.

So what is there so far? It’s all new code written in the C language and has about 30k lines of code in 150+ files so far. It’s fully threaded in a way that should make it very scalable on many cores/cpu’s. More about the threading in a future post. The code is heavily unit tested, which really helps a lot in preventing and tracing bugs.

Right now it’s limited to being an inline IDS/IPS, using the libnetfilter_queue interface in Linux to acquire and verdict packets. The packet input and verdict subsystem is very modular (I learned a lot from the mess we created in Snort_inline, where we supported 3 types of inline packet capture methods, creating a true #ifdef hell). It has working protocol decoders for IPv4 and IPv6, TCP and UDP. It has a flow engine, a detection engine and output plugins.

For rules/signatures it currently only supports the Snort signature syntax, and loads about 70% of the current VRT and Emerging Threats signatures out there. The biggest thing missing is support for the flowbits option, which is used in a lot of the sigs. It has basic HTTP parsing, enabling at least uri matching.

A lot of things are missing too. For example there is fragment handling, TCP stream state tracking, TCP stream reassembly, a pcap mode, portscan detection, a flowbits like function, normalization, etc, etc.

There are a lot of plans and ideas, for example having output pipes for configurable captured network data. It’s already possible to capture for example a user agent in a rule and match on that captured data. I think it would be very useful to be able to have some pipe to an external program that receives just the user agents and does something with them. Many many more ideas and usecases exist and I hope to write about that more at a later stage.

The most interesting about writing this code is that every time I’m working on some part, I’m getting more and more ideas about possibilities for improvements, optimizations and such. I intent to share those here on my blog from now. Also, I intent to write about the various parts of the code I wrote already. So stay tuned!

Tags: , ,
Posted in ids, IPS, oisf | 2 Comments »

Available for contract work

Monday, January 5th, 2009

This year there will be a lot of work that needs to be done for the Open Infosec Foundation. And like I wrote a few days ago, a lot of work is already being done. However, most of it is unpaid at this time as it will be some months before our funding comes in. So at least until then I’m available and looking for contract work.

For the last two years I’ve been doing work as a contractor in the (open source) security field. My experience is mostly in coding in C and Perl, primarily on Snort and Snort_inline. Recently I created the (Perl language) SidReporter program for Emerging Threats. Areas I worked in: IPv6 IDS/IPS coding, signature writing, Web Application Firewalls, threading, bandwidth accounting, and more…

Checkout my LinkedIn profile for more info. My resume is available on request.

If you have some work or know someone that does, please let me know!

Tags: , , , , , , ,
Posted in ids, IPS, IPv6, oisf, Personal, SidReporter, Snort, Snort_inline | 1 Comment »

Looking forward to 2009: Open Infosec Foundation

Monday, December 29th, 2008

The year 2008 was an exciting year to me. The biggest thing going on the infosec side was the formation of the Open Infosec Foundation. We’ve been working on it behind the scenes for more than a year now, and it’s cool that we’ve finally announced our plans. Of course, the work is just getting started. Next year, we expect to finalize our foundation setup. We’re working with the Software Freedom Law Center for setting up the foundation charter and consortium rules. While the US government is funding us initially, we hope the consortium will guarantee our long term funding. We are talking to some interesting companies already, both big and small.

The last year I’ve been working on a prototype of the engine we’re building as well. It’s private for now as the foundation licensing terms & conditions haven’t been determined yet. I’m writing it mostly to learn. While I’ve been working as a developer on the Snort_inline project for a number of years already and as a contractor on several Snort related projects, I never learned so much about IDS/IPS technology as I’m doing now. The prototype may or may not be used (partly) for the engine once we got our feature list complete. We’ll see about that when the time is there. I plan to blog more about this codebase in the new year.

In 2008 we had our first brainstorming session, and to us it was very successful. In 2009 we’re hoping to do a few more. Stay tuned for the dates and places. I hope we can continue our feature discussions in the new year and give the foundation further shape. And don’t forget to suggest us a name for the engine… “OISF engine” just doesn’t sound cool enough! ;-)

Tags: , ,
Posted in ids, IPS, oisf | 1 Comment »

Open Infosec Foundation founded!

Saturday, October 18th, 2008

Last week Matt Jonkman announced the formation of the Open Infosec Foundation. This foundation has been grant funded to create a new open source IDS/IPS engine. Together with Will Metcalf and of course Matt himself, I will be working on this. We want this to be a real community effort where there is a role for everyone in the infosec community. Developers, admins, vendors, goverments, research, education, everyone. There is a lot of work ahead, but that should be great fun and very inspiring. So far things are interesting already. The discussion mailinglist is growing rapidly with many ppl from the community and industry. A #oisf IRC channel was created today on freenode. Join us there to participate in discussion about this project!

Tags: , ,
Posted in ids, IPS, oisf | No Comments »

Snort_inline and out of order packets

Monday, July 30th, 2007

In Snort_inline’s stream4 modifications, one of the changes is that out of order TCP packets are treated differently from unmodified stream4. This can cause some new alerts to appear and some unexpected behaviour. So I’ll try to explain what happens here.

First of all let me explain quickly what out of order packets are. To put it simple, TCP packets are send out by the source host in a specific order but can arrive in a different order at the destination. Packetloss, link saturation, routing issues are among many things that can cause this. A Snort_inline specific issue is that when Snort_inline can’t keep up with the packets it needs to process, it will drop packets which causes packetloss. These packets will then have to be resent by the sending host.

Out of order packets become a problem when dealing with stream reassembly. Stream reassembly basically is putting all data from the packets in the right order to get the original data as it was sent. We can’t do stream reassembly if we don’t have all packets. Unmodified stream4 basically ignores gaps in the stream. Designed for passive listening for traffic, it has to deal with packetloss differently than Snort_inline.

Next, some definitions of this functionality in Snort_inline. Out-of-order packets: The number of packets that we have in queue that are out of order for a stream. This means they have a higher sequence number than the next in-sequence packet we are expecting. Out-of-order bytes: The number of bytes of the combined data of the out-of-order packets in the stream. Sequence number hole: A gap between two packets, that can be closed by one or more missing packets.

To prevent Snort_inline from using to much memory on bad connections or when an attacker sends lots of out of order packets, Snort_inline can enforce limits to protect itself. Snort_inline can even force a stream to be completely in-order by dropping all packets that are out of order. Sadly, this has a bad effect on the performance of the connections, so you can set certain limits that balance between performance and protection.

When Snort_inline hits these limits, it will (optionally) fire alerts that look like this:

(spp_stream4) TCP out-of-order packets limit reached for stream
(spp_stream4) TCP out-of-order bytes limit reached for stream
(spp_stream4) TCP sequence number holes limit reached for stream

You can disable the alerts by adding the following option to the preprocessor stream4 line: disable_ooo_alerts. The limits themselves can be adjusted by using the following options: max_seq_holes 2, max_ooo_pkts 25, max_ooo_bytes 7000. These are the values I currently use on my home gateway. I got the idea of implementing these limits from this paper by Vern Paxson. However, it seems to me that his suggestion that at max one sequence hole per stream (even per host) was a bit optimistic. Maybe DSL has more packetloss than the university links he studied.

By default Snort_inline uses the settings that were chosen a bit randomly, so they may not fit your usage. Like with the wscaling, please let me know in a comment what values you use!

Tags: , , , , , ,
Posted in IPS, Snort_inline, TCP | 1 Comment »

Update on using realtime blacklists with ModSecurity

Thursday, March 1st, 2007

A few days ago I posted a blog article about stopping comment spam with ModSecurity using realtime blacklists (rbl). While the approach was working, I noted having problems with rules when I tried to match on POST methods in HTTP requests.

Luckily, ModSecurity creator Ivan Ristic was quick to point out where the problem is. I’m using the Core Ruleset for ModSecurity, and one thing that ruleset does is use the ‘lowercase’ transformation. This converts all text from arguments to lowercase, so my ^POST$ match would never be able to match. So like Ivan suggested, using ^post$ solved this part.

Next Ivan pointed out a weakness in the rules. My rules looked for /blog/wp-comment-post.php, and would be easily evaded by just using /blog//wp-comment-post.php. He suggested using the ‘normalisePath’ transformation. I did this, but I also slightly changed the rules to not look for the /blog/ part at all (maybe this makes normalisePath useless, but I decided to rather be safe than sorry).

The rules I’m using now look like this:

SecRule REQUEST_METHOD “^post$” “log,deny,chain,msg:’LOCAL comment spammer at rbl list.dsbl.org’”
SecRule REQUEST_URI “wp-(comments-post|trackback)\.php$” “chain,t:normalisePath”
SecRule REMOTE_ADDR “@rbl list.dsbl.org”

SecRule REQUEST_METHOD “^post$” “log,deny,chain,msg:’LOCAL comment spammer at rbl bl.spamcop.net’”
SecRule REQUEST_URI “wp-(comments-post|trackback)\.php$” “chain,t:normalisePath”
SecRule REMOTE_ADDR “@rbl bl.spamcop.net”

SecRule REQUEST_METHOD “^post$” “log,deny,chain,msg:’LOCAL comment spammer at rbl sbl-xbl.spamhaus.org’”
SecRule REQUEST_URI “wp-(comments-post|trackback)\.php$” “chain,t:normalisePath”
SecRule REMOTE_ADDR “@rbl sbl-xbl.spamhaus.org”

Thanks a lot Ivan Ristic for your comments!

Tags: , , ,
Posted in IPS, ModSecurity, Web | No Comments »

Blocking comment spam using ModSecurity and realtime blacklists

Friday, February 23rd, 2007

Spammers are known to use compromised hosts from all over the world to send their messages. Many people are blocking or scoring email spam based on realtime blacklist (rbl), which contain ipaddresses of these known bad hosts. In my experience this works fairly well for email. A while ago I noticed in the ModSecurity documentation for version 2.0 that ModSecurity features an operator called rbl, that can be used to check the ipaddress of a visitor with a rbl. So I decided to see if I could use the realtime blacklists to prevent comment spam on my blog. Turns out this works great! In this post I’ll show how to get it working.
(more…)

Tags: , ,
Posted in IPS, ModSecurity, Web | 10 Comments »

« Older Entries
Newer Entries »