Archive for the ‘IPS’ Category

« Older Entries

Setting up Suricata 0.9.0 for initial use on Ubuntu Lucid 10.04

Monday, May 10th, 2010

The last few days I blogged about compiling Suricata in IDS and IPS mode. Today I’ll write about how to set it up for first use.

Starting with Suricata 0.9.0 the engine can run as an unprivileged user. For this create a new user called “suricata”.

useradd --no-create-home --shell /bin/false --user-group --comment “Suricata IDP account” suricata

This command will create a user and group called “suricata”. It will be unable to login as the shell is set to /bin/false.

The next thing to do is creating a configuration directory. Create /etc/suricata/ and copy the suricata.yaml example config into it. The example configuration can be found in the source archive you used to build Suricata:

mkdir /etc/suricata
cp /path/to/suricata-0.9.0/suricata.yaml /etc/suricata/
cp /path/to/suricata-0.9.0/classification.config /etc/suricata/

Next, create the log directory.

mkdir /var/log/suricata

The log directory needs to be writable for the user and group “suricata”, so change the ownership:

chown suricata:suricata /var/log/suricata

The last step I’ll be describing here is retrieving an initial ruleset. The 2 main rulesets you can use are Emerging Threats (ET) and Sourcefire’s VRT ruleset. Since putting VRT to use is a little bit more complicated I’ll be focussing on ET here.

First, download the emerging rules:

wget http://www.emergingthreats.net/rules/emerging.rules.tar.gz

Go to /etc/suricata/ and extract the rules archive:

cd /etc/suricata/
tar xzvf /path/to/emerging.rules.tar.gz

There is a lot more to rules, such as tuning and staying updated, but thats beyond the scope of this post.

Suricata is now ready to be started:

suricata -c /etc/suricata/suricata.yaml -i eth0 --user suricata --group suricata

If all is setup properly, Suricata will tell you it is now running:

[2087] 9/5/2010 — 18:17:47 – (tm-threads.c:1362) (TmThreadWaitOnThreadInit) — all 8 packet processing threads, 3 management threads initialized, engine started.

There are 3 log files in /var/log/suricata that will be interesting to monitor:

- stats.log: displays statistics on packets, tcp sessions etc.
- fast.log: a alerts log similar to Snort’s fast log.
- http.log: displays HTTP requests in a Apache style format.

This should get you going. There is a lot more to deploying Suricata that I plan to blog on later.

Tags: , , , ,
Posted in IPS, Suricata, Ubuntu, ids, oisf | 1 Comment »

Compiling Suricata 0.9.0 in Ubuntu Lucid 10.04 in IPS (inline) mode

Friday, May 7th, 2010

Note: the difference with the 0.8.2 post is that addition of libcap-ng-dev. This allows Suricata to run as an unprivileged user.

Here is how to compile Suricata 0.9.0 in inline mode on Ubuntu Lucid 10.04.

First, make sure you have the “universe” repository enabled. Go to the System menu, Administration, Software Sources. There enable “Community-maintained Open Source Software (universe)”. If you’re not running a gui, edit /etc/apt/sources.list and enable the universe repository there. Don’t forget doing an “apt-get update”.

Install the following packages needed to build Suricata: libpcre3-dev libpcap-dev libyaml-dev zlib1g-dev libnfnetlink-dev libnetfilter-queue-dev libnet1-dev libcap-ng-dev.

apt-get install libpcre3-dev libpcap-dev libyaml-dev zlib1g-dev libnfnetlink-dev libnetfilter-queue-dev libnet1-dev libcap-ng-dev

Download Suricata 0.9.0 here

Extract the suricata-0.9.0.tar.gz file as follows:

tar xzvf suricata-0.9.0.tar.gz

Enter the extracted directory suricata-0.9.0.

Run “./configure –enable-nfqueue”
If “./configure –enable-nfqueue” was succesful, run “make”
If “make” was succesful, run “sudo make install”
Except for Suricata itself, the build process installed “libhtp”. For that to work properly, run “ldconfig”.

Run “suricata -V” and it should report version 0.9.0.

To use Suricata in inline mode, pass -q <queue id> to the command line. Example:

suricata -c /etc/suricata/suricata.yaml -q 0

Tags: , , ,
Posted in IPS, Suricata, Ubuntu, oisf | 5 Comments »

Suricata 0.9.0 released

Friday, May 7th, 2010

Yesterday we released we first release candidate for our upcoming 1.0 release of Suricata. See the announcement on the OISF site here.

Most notable changes are the following new features:

- Support for the http_headers keyword was added
- libhtp was updated to version 0.2.3
- Privilege dropping using libcap-ng is now supported
- Proper support for “pass” rules was added
- Inline mode for Windows was added

Go get the release here: http://www.openinfosecfoundation.org/download/suricata-0.9.0.tar.gz

Tags: , , , , ,
Posted in IPS, Suricata, ids, oisf | No Comments »

Compiling Suricata 0.8.2 in Ubuntu Lucid 10.04 in IPS (inline) mode

Saturday, May 1st, 2010

Yesterday I wrote about how to compile and install Suricata 0.8.2 as an IDS on Ubuntu Lucid 10.04, today I’ll explain the steps to compile and install it as an IPS. In IPS mode the engine runs in inline mode. This means that it gets it’s packets from netfilter and sets a verdict on them after inspecting them. This way we can drop packets that trigger the rules.

First, make sure you have the “universe” repository enabled. Go to the System menu, Administration, Software Sources. There enable “Community-maintained Open Source Software (universe)”. If you’re not running a gui, edit /etc/apt/sources.list and enable the universe repository there. Don’t forget doing an “apt-get update”.

Install the following packages needed to build Suricata: libpcre3-dev libpcap-dev libyaml-dev zlib1g-dev libnfnetlink-dev libnetfilter-queue-dev libnet1-dev.

apt-get install libpcre3-dev libpcap-dev libyaml-dev zlib1g-dev libnfnetlink-dev libnetfilter-queue-dev libnet1-dev

Download Suricata 0.8.2 here

Extract the suricata-0.8.2.tar.gz file as follows:

tar xzvf suricata-0.8.2.tar.gz

Enter the extracted directory suricata-0.8.2.

Run “./configure –enable-nfqueue”
If “./configure –enable-nfqueue” was succesful, run “make”
If “make” was succesful, run “sudo make install”
Except for Suricata itself, the build process installed “libhtp”. For that to work properly, run “ldconfig”.

Run “suricata -V” and it should report version 0.8.2.

To use Suricata in inline mode, pass -q <queue id> to the command line. Example:

suricata -c /etc/suricata/suricata.yaml -q 0

Tags: , , ,
Posted in IPS, Suricata, Ubuntu, oisf | 1 Comment »

Suricata 0.8.2 released

Monday, April 19th, 2010

Today the OISF development team released 0.8.2 of the Suricata IDS/IPS engine. I feel this is definitely the best release so far. Read the announcement here. In short, stability was improved, memory footprint reduced, performance improved and new features were added.

One of the tools we used to help improve the engine is a fuzzer created by Will Metcalf, our QA lead. In short, the script takes a pcap file, runs it through editcap (part of wireshark) altering a number of random bytes, then feeds the altered pcap file to Suricata. This resulted in many interesting corner cases. Naturally the script makes sure you don’t forget to enable “ulimit -c unlimited” and such :) More on that script can be found on Will’s blog node5.

For the next period we’ll be working on resolving a number of open issues. There are still a number of improvements we need to make to the relation between our app layer decoding modules and our detection engine. Next to this we’re still missing support for a number of rule keywords, such as asn1 and http_headers. We’re also working on getting our CUDA accelaration into a more usable shape. This release improved it slightly, by making it work on x86_64, but it’s still not useful in production environments.

So as usual, enough to do! Meanwhile, we’re looking for feedback on our release!

Tags: , ,
Posted in IPS, Suricata, ids, oisf | 1 Comment »

Suricata 0.8.1 released

Saturday, February 20th, 2010

Yesterday the OISF development team released Suricata 0.8.1. This release is much improved from our December 31st release. It is way more stable, performs better and has more features. Thanks to the now included HTP library we have much better HTTP handling. The stream engine has seen massive improvements. Initial experimental CUDA code has been added. Initial Win32 support has been added. We’ve added number of missing rule keywords. Many bugs were fixed.

Personally I’m very excited about the help we have gotten from the community. Quite a few patches from community members were applied in this release. Thanks everyone!

Next week the OISF team and a number of experts are meeting up in Istanbul. We’ll be working on crunching a number of technical challenges, sharing ideas and we will start our brainstorming on future development. If you have any ideas about where you think IDS/IPS should go, please let us know so we can discuss it and possibly include it in our future plans.

Tags: , , ,
Posted in IPS, Suricata, ids, oisf | No Comments »

OISF IDS/IPS engine prototype intro

Wednesday, January 7th, 2009

For over a year I’ve been working on a prototype implementation of a new IDS/IPS engine for the Open Infosec Foundation. This is not necessarily going to be the engine we’ll be using in OISF, although it’s likely that at least some of the code will be used. Discussions about features for the engine are still ongoing (wiki, list), once that settles down we’ll see whats usable and whats not. In the worst case I still think many parts like hashing functions, pattern matcher implementations, protocol decoders, etc can be used.

So what is there so far? It’s all new code written in the C language and has about 30k lines of code in 150+ files so far. It’s fully threaded in a way that should make it very scalable on many cores/cpu’s. More about the threading in a future post. The code is heavily unit tested, which really helps a lot in preventing and tracing bugs.

Right now it’s limited to being an inline IDS/IPS, using the libnetfilter_queue interface in Linux to acquire and verdict packets. The packet input and verdict subsystem is very modular (I learned a lot from the mess we created in Snort_inline, where we supported 3 types of inline packet capture methods, creating a true #ifdef hell). It has working protocol decoders for IPv4 and IPv6, TCP and UDP. It has a flow engine, a detection engine and output plugins.

For rules/signatures it currently only supports the Snort signature syntax, and loads about 70% of the current VRT and Emerging Threats signatures out there. The biggest thing missing is support for the flowbits option, which is used in a lot of the sigs. It has basic HTTP parsing, enabling at least uri matching.

A lot of things are missing too. For example there is fragment handling, TCP stream state tracking, TCP stream reassembly, a pcap mode, portscan detection, a flowbits like function, normalization, etc, etc.

There are a lot of plans and ideas, for example having output pipes for configurable captured network data. It’s already possible to capture for example a user agent in a rule and match on that captured data. I think it would be very useful to be able to have some pipe to an external program that receives just the user agents and does something with them. Many many more ideas and usecases exist and I hope to write about that more at a later stage.

The most interesting about writing this code is that every time I’m working on some part, I’m getting more and more ideas about possibilities for improvements, optimizations and such. I intent to share those here on my blog from now. Also, I intent to write about the various parts of the code I wrote already. So stay tuned!

Tags: , ,
Posted in IPS, ids, oisf | 2 Comments »

Available for contract work

Monday, January 5th, 2009

This year there will be a lot of work that needs to be done for the Open Infosec Foundation. And like I wrote a few days ago, a lot of work is already being done. However, most of it is unpaid at this time as it will be some months before our funding comes in. So at least until then I’m available and looking for contract work.

For the last two years I’ve been doing work as a contractor in the (open source) security field. My experience is mostly in coding in C and Perl, primarily on Snort and Snort_inline. Recently I created the (Perl language) SidReporter program for Emerging Threats. Areas I worked in: IPv6 IDS/IPS coding, signature writing, Web Application Firewalls, threading, bandwidth accounting, and more…

Checkout my LinkedIn profile for more info. My resume is available on request.

If you have some work or know someone that does, please let me know!

Tags: , , , , , , ,
Posted in IPS, IPv6, Personal, SidReporter, Snort, Snort_inline, ids, oisf | 1 Comment »

Looking forward to 2009: Open Infosec Foundation

Monday, December 29th, 2008

The year 2008 was an exciting year to me. The biggest thing going on the infosec side was the formation of the Open Infosec Foundation. We’ve been working on it behind the scenes for more than a year now, and it’s cool that we’ve finally announced our plans. Of course, the work is just getting started. Next year, we expect to finalize our foundation setup. We’re working with the Software Freedom Law Center for setting up the foundation charter and consortium rules. While the US government is funding us initially, we hope the consortium will guarantee our long term funding. We are talking to some interesting companies already, both big and small.

The last year I’ve been working on a prototype of the engine we’re building as well. It’s private for now as the foundation licensing terms & conditions haven’t been determined yet. I’m writing it mostly to learn. While I’ve been working as a developer on the Snort_inline project for a number of years already and as a contractor on several Snort related projects, I never learned so much about IDS/IPS technology as I’m doing now. The prototype may or may not be used (partly) for the engine once we got our feature list complete. We’ll see about that when the time is there. I plan to blog more about this codebase in the new year.

In 2008 we had our first brainstorming session, and to us it was very successful. In 2009 we’re hoping to do a few more. Stay tuned for the dates and places. I hope we can continue our feature discussions in the new year and give the foundation further shape. And don’t forget to suggest us a name for the engine… “OISF engine” just doesn’t sound cool enough! ;-)

Tags: , ,
Posted in IPS, ids, oisf | 1 Comment »

Open Infosec Foundation founded!

Saturday, October 18th, 2008

Last week Matt Jonkman announced the formation of the Open Infosec Foundation. This foundation has been grant funded to create a new open source IDS/IPS engine. Together with Will Metcalf and of course Matt himself, I will be working on this. We want this to be a real community effort where there is a role for everyone in the infosec community. Developers, admins, vendors, goverments, research, education, everyone. There is a lot of work ahead, but that should be great fun and very inspiring. So far things are interesting already. The discussion mailinglist is growing rapidly with many ppl from the community and industry. A #oisf IRC channel was created today on freenode. Join us there to participate in discussion about this project!

Tags: , ,
Posted in IPS, ids, oisf | No Comments »

« Older Entries