Setting up Suricata 0.9.0 for initial use on Ubuntu Lucid 10.04

May 10th, 2010

The last few days I blogged about compiling Suricata in IDS and IPS mode. Today I’ll write about how to set it up for first use.

Starting with Suricata 0.9.0 the engine can run as an unprivileged user. For this create a new user called “suricata”.

useradd --no-create-home --shell /bin/false --user-group --comment “Suricata IDP account” suricata

This command will create a user and group called “suricata”. It will be unable to login as the shell is set to /bin/false.

The next thing to do is creating a configuration directory. Create /etc/suricata/ and copy the suricata.yaml example config into it. The example configuration can be found in the source archive you used to build Suricata:

mkdir /etc/suricata
cp /path/to/suricata-0.9.0/suricata.yaml /etc/suricata/
cp /path/to/suricata-0.9.0/classification.config /etc/suricata/

Next, create the log directory.

mkdir /var/log/suricata

The log directory needs to be writable for the user and group “suricata”, so change the ownership:

chown suricata:suricata /var/log/suricata

The last step I’ll be describing here is retrieving an initial ruleset. The 2 main rulesets you can use are Emerging Threats (ET) and Sourcefire’s VRT ruleset. Since putting VRT to use is a little bit more complicated I’ll be focussing on ET here.

First, download the emerging rules:

wget http://www.emergingthreats.net/rules/emerging.rules.tar.gz

Go to /etc/suricata/ and extract the rules archive:

cd /etc/suricata/
tar xzvf /path/to/emerging.rules.tar.gz

There is a lot more to rules, such as tuning and staying updated, but thats beyond the scope of this post.

Suricata is now ready to be started:

suricata -c /etc/suricata/suricata.yaml -i eth0 --user suricata --group suricata

If all is setup properly, Suricata will tell you it is now running:

[2087] 9/5/2010 — 18:17:47 – (tm-threads.c:1362) (TmThreadWaitOnThreadInit) — all 8 packet processing threads, 3 management threads initialized, engine started.

There are 3 log files in /var/log/suricata that will be interesting to monitor:

- stats.log: displays statistics on packets, tcp sessions etc.
- fast.log: a alerts log similar to Snort’s fast log.
- http.log: displays HTTP requests in a Apache style format.

This should get you going. There is a lot more to deploying Suricata that I plan to blog on later.

Compiling Suricata 0.9.0 in Ubuntu Lucid 10.04 in IPS (inline) mode

May 7th, 2010

Note: the difference with the 0.8.2 post is that addition of libcap-ng-dev. This allows Suricata to run as an unprivileged user.

Here is how to compile Suricata 0.9.0 in inline mode on Ubuntu Lucid 10.04.

First, make sure you have the “universe” repository enabled. Go to the System menu, Administration, Software Sources. There enable “Community-maintained Open Source Software (universe)”. If you’re not running a gui, edit /etc/apt/sources.list and enable the universe repository there. Don’t forget doing an “apt-get update”.

Install the following packages needed to build Suricata: libpcre3-dev libpcap-dev libyaml-dev zlib1g-dev libnfnetlink-dev libnetfilter-queue-dev libnet1-dev libcap-ng-dev.

apt-get install libpcre3-dev libpcap-dev libyaml-dev zlib1g-dev libnfnetlink-dev libnetfilter-queue-dev libnet1-dev libcap-ng-dev

Download Suricata 0.9.0 here

Extract the suricata-0.9.0.tar.gz file as follows:

tar xzvf suricata-0.9.0.tar.gz

Enter the extracted directory suricata-0.9.0.

Run “./configure –enable-nfqueue”
If “./configure –enable-nfqueue” was succesful, run “make”
If “make” was succesful, run “sudo make install”
Except for Suricata itself, the build process installed “libhtp”. For that to work properly, run “ldconfig”.

Run “suricata -V” and it should report version 0.9.0.

To use Suricata in inline mode, pass -q <queue id> to the command line. Example:

suricata -c /etc/suricata/suricata.yaml -q 0

Compiling Suricata 0.9.0 in Ubuntu Lucid 10.04 in IDS mode

May 7th, 2010

Note: the difference with the 0.8.2 post is that addition of libcap-ng-dev. This allows Suricata to run as an unprivileged user.

Ubuntu Lucid 10.04 LTS looks like a good platform for running an IDS on. It’s up to date and has long term support. Here is how to compile and install Suricata 0.9.0 on it.

Install the following packages needed to build Suricata: libpcre3-dev libpcap-dev libyaml-dev zlib1g-dev libcap-ng-dev.

apt-get install libpcre3-dev libpcap-dev libyaml-dev zlib1g-dev libcap-ng-dev

Download Suricata 0.9.0 here

Extract the suricata-0.9.0.tar.gz file as follows:

tar xzvf suricata-0.9.0.tar.gz

Enter the extracted directory suricata-0.9.0.

Run “./configure”
Note that you may get a warning about libnet 1.1 that is missing. You can ignore that, it’s only used in IPS/inline mode currently.
If “./configure” was succesful, run “make”
If “make” was succesful, run “sudo make install”
Except for Suricata itself, the build process installed “libhtp”. For that to work properly, run “ldconfig”.

Run “suricata -V” and it should report version 0.9.0.

To use Suricata in IDS mode, pass -i to the command line. Example

suricata -c /etc/suricata/suricata.yaml -i eth0

Suricata 0.9.0 released

May 7th, 2010

Yesterday we released we first release candidate for our upcoming 1.0 release of Suricata. See the announcement on the OISF site here.

Most notable changes are the following new features:

- Support for the http_headers keyword was added
- libhtp was updated to version 0.2.3
- Privilege dropping using libcap-ng is now supported
- Proper support for “pass” rules was added
- Inline mode for Windows was added

Go get the release here: http://www.openinfosecfoundation.org/download/suricata-0.9.0.tar.gz

Compiling Suricata 0.8.2 in Ubuntu Lucid 10.04 in IPS (inline) mode

May 1st, 2010

Yesterday I wrote about how to compile and install Suricata 0.8.2 as an IDS on Ubuntu Lucid 10.04, today I’ll explain the steps to compile and install it as an IPS. In IPS mode the engine runs in inline mode. This means that it gets it’s packets from netfilter and sets a verdict on them after inspecting them. This way we can drop packets that trigger the rules.

First, make sure you have the “universe” repository enabled. Go to the System menu, Administration, Software Sources. There enable “Community-maintained Open Source Software (universe)”. If you’re not running a gui, edit /etc/apt/sources.list and enable the universe repository there. Don’t forget doing an “apt-get update”.

Install the following packages needed to build Suricata: libpcre3-dev libpcap-dev libyaml-dev zlib1g-dev libnfnetlink-dev libnetfilter-queue-dev libnet1-dev.

apt-get install libpcre3-dev libpcap-dev libyaml-dev zlib1g-dev libnfnetlink-dev libnetfilter-queue-dev libnet1-dev

Download Suricata 0.8.2 here

Extract the suricata-0.8.2.tar.gz file as follows:

tar xzvf suricata-0.8.2.tar.gz

Enter the extracted directory suricata-0.8.2.

Run “./configure –enable-nfqueue”
If “./configure –enable-nfqueue” was succesful, run “make”
If “make” was succesful, run “sudo make install”
Except for Suricata itself, the build process installed “libhtp”. For that to work properly, run “ldconfig”.

Run “suricata -V” and it should report version 0.8.2.

To use Suricata in inline mode, pass -q <queue id> to the command line. Example:

suricata -c /etc/suricata/suricata.yaml -q 0

Compiling Suricata 0.8.2 in Ubuntu Lucid 10.04 in IDS mode

April 30th, 2010

The newly released Ubuntu Lucid 10.04 LTS looks like a good platform for running an IDS on. It’s up to date and has long term support. Here is how to compile and install Suricata 0.8.2 on it.

Install the following packages needed to build Suricata: libpcre3-dev libpcap-dev libyaml-dev zlib1g-dev.

apt-get install libpcre3-dev libpcap-dev libyaml-dev zlib1g-dev

Download Suricata 0.8.2 here

Extract the suricata-0.8.2.tar.gz file as follows:

tar xzvf suricata-0.8.2.tar.gz

Enter the extracted directory suricata-0.8.2.

Run “./configure”
Note that you may get a warning about libnet 1.1 that is missing. You can ignore that, it’s only used in IPS/inline mode currently.
If “./configure” was succesful, run “make”
If “make” was succesful, run “sudo make install”
Except for Suricata itself, the build process installed “libhtp”. For that to work properly, run “ldconfig”.

Run “suricata -V” and it should report version 0.8.2.

To use Suricata in IDS mode, pass -i to the command line. Example

suricata -c /etc/suricata/suricata.yaml -i eth0

Removing Trac ticket comment spam in Debian Lenny

April 23rd, 2010

The Vuurmuur website runs Trac and overall I’m pretty happy with it. The only thing that Trac doesn’t do well, is dealing with spammers. Spammers target Trac a lot, so that’s a real problem.

To prevent spammers from making it through, I run Scallywhack and a number of custom ModSecurity rules. So far, spams only made it through as new tickets in the ticket tracker, so I installed the TicketDeletePlugin.

Yesterday, I saw the first spam as a comment to an existing and valid ticket. Like tickets themselves, ticket comments can not be removed by Trac by default. Luckily, upstream Trac seems to have fixed this. I’m running Debian’s version of Trac 0.11.1 however, so I decided to patch that. The patches in the Trac ticket #454 didn’t apply cleanly, so I had to patch it manually. To save others the work, it’s available here: http://www.inliniac.net/files/trac_0.11.1-debian-comment_edit.patch

To use it, make a copy of your /usr/share/pyshared/trac directory.
Next, go into the trac directory and run the command:
patch -p1 < /path/to/trac_0.11.1-debian-comment_edit.patch

After this, each comment in the comment system will have a “edit” button and you can remove the spam message content. It’s not possible to remove the entire comment, but this works for me.

Suricata 0.8.2 released

April 19th, 2010

Today the OISF development team released 0.8.2 of the Suricata IDS/IPS engine. I feel this is definitely the best release so far. Read the announcement here. In short, stability was improved, memory footprint reduced, performance improved and new features were added.

One of the tools we used to help improve the engine is a fuzzer created by Will Metcalf, our QA lead. In short, the script takes a pcap file, runs it through editcap (part of wireshark) altering a number of random bytes, then feeds the altered pcap file to Suricata. This resulted in many interesting corner cases. Naturally the script makes sure you don’t forget to enable “ulimit -c unlimited” and such :) More on that script can be found on Will’s blog node5.

For the next period we’ll be working on resolving a number of open issues. There are still a number of improvements we need to make to the relation between our app layer decoding modules and our detection engine. Next to this we’re still missing support for a number of rule keywords, such as asn1 and http_headers. We’re also working on getting our CUDA accelaration into a more usable shape. This release improved it slightly, by making it work on x86_64, but it’s still not useful in production environments.

So as usual, enough to do! Meanwhile, we’re looking for feedback on our release!

Book review: Magnus Mischel – ModSecurity 2.5

March 31st, 2010

It’s been quite a while since I received my review copy of Magnus Mischel’s ModSecurity book titled “ModSecurity 2.5″ but I finally found the time to read it and write up my review. As the title suggest it’s a book about the ModSecurity Web Application Firewall (WAF) module for Apache and about version 2.5 of it specifically. There are some books about the 1.x series of ModSecurity. It’s great that there is a book about the 2.x ModSecurity series now as ModSecurity 2.x is very different from the 1.x series.

The ModSecurity module is very powerful but also very complex. It’s pretty trivial to add a few rules blocking some attacks, but when trying to protect large web applications such as OWA things get complicated very quickly. But even with a smaller system like WordPress I found that finding the right approach is not trivial. While there is an online manual and there are an array of blogposts (some written by me even), a good overview of ModSecurity’s features and how to really deploy it properly and effectively in complex environments is lacking. This is what I hoped to find in this book.

Giving this expectation the book slightly disappointed me. But let me start out with what I liked about the book.

The book gives a broad overview of how ModSecurity can be used. It deals with the obvious parts like compilation, installation and setting up, but also handles more interesting parts like virtual patching, performance profiling, the difference between “positive” and “negative” security approaches, REMO (a web based open source ModSecurity rule editor) and more. I learned quite a bit here, for example about directives to deal with credit card numbers.

Where it falls short is mostly in the lack of depth. It touches a lot of subjects, but most of them only pretty briefly. Next to this a view chapters could be organized a little better, especially in the first couple of chapters. I think what would really improve this book is adding the approach done by my favorite ModSecurity book so far, Ryan C. Barnetts “Preventing Web Attacks With Apache”. In that book a flawed application is introduced (Buggy Bank) and much time is spend on explaining how things are broken and where ModSecurity can and cannot help.

My verdict is that “ModSecurity 2.5″ is a good introductionary book into ModSecurity, but that it’s missing some depth to be much more than that. Being someone that has quite a bit of ModSecurity experience, including writing pretty complex rulesets, I had hoped for more help on dealing with those. But all being said, I still recommend this book to anyone that is in need of a good introduction into ModSecurity. I recommend picking up Ryan C. Barnetts “Preventing Web Attacks With Apache” book alongside with it, even though it deals with ModSecurity 1.x. I think together they will provide enough depth to deal with a real world environment.

On a final note I’d like to mention that Ivan Ristic, the original ModSecurity creator, has also written a new book on ModSecurity. I haven’t read that yet, but Ivan’s first book was excellent.

Suricata has experimental CUDA support

February 20th, 2010

One area of interest in the development of Suricata is hardware acceleration. Using the GPU is particularly interesting, as they are cheap and widely available. We’ve been looking at using the GPU to speed up pattern matching as a first step. Since OpenCL promises to be a cross platform multi vendor API for doing this we first looked at OpenCL. But we were never able to get something stable out of it, not on the NVIDIA drivers in Linux anyway. As that didn’t go anywhere we decided to use CUDA for the time being. CUDA obviously is NVIDIA only. Once we have CUDA fully running we may revisit OpenCL or look at other implementations like AMD/ATI’s stream API.

What we have so far is a implementation our 2 gram SBNDM pattern matcher algorithm in CUDA. The detection thread(s) currently send packets one by one to a central dispatcher thread that controls the GPU. This setup is far from ideal performance wise, but our first goal was to get it working at all. Currently on my desktop CUDA actually slows things down.

In the next weeks and months we plan to do some redesigning of the CUDA implementation and it’s integration into the engine. We plan to send the packets in batches to the dispatcher thread right after the decoders have determined what the payload portion of a packet is. The (separate) detection thread(s) can then process the results of the GPU when they get to a packet. By using the CUDA scanning async like this we hope that we can reduce the costs of the transfer of packets from and to the card.

Currently the code in the tree can be activated by passing the “–enable-cuda” option to ./configure. Next, in the configuration file enable the cuda pattern matcher by setting the “mpm-algo” option to “b2g_cuda”. As a first test, run the CUDA unittests (assuming you enabled the building of the unittests too) by using “suricata -uUCuda”. Please note that currently running all unittests will fail if CUDA is enabled.

The code is only tested on 32bit Linux at the moment. There are some issues with 64bit that we’re resolving right now. We’re expecting to be continuously updating this code, so be sure to work with the most current version of the git repo all the time!

Let us know your experiences!