April 30th, 2010
The newly released Ubuntu Lucid 10.04 LTS looks like a good platform for running an IDS on. It’s up to date and has long term support. Here is how to compile and install Suricata 0.8.2 on it.
Install the following packages needed to build Suricata: libpcre3-dev libpcap-dev libyaml-dev zlib1g-dev.
apt-get install libpcre3-dev libpcap-dev libyaml-dev zlib1g-dev
Download Suricata 0.8.2 here
Extract the suricata-0.8.2.tar.gz file as follows:
tar xzvf suricata-0.8.2.tar.gz
Enter the extracted directory suricata-0.8.2.
Run “./configure”
Note that you may get a warning about libnet 1.1 that is missing. You can ignore that, it’s only used in IPS/inline mode currently.
If “./configure” was succesful, run “make”
If “make” was succesful, run “sudo make install”
Except for Suricata itself, the build process installed “libhtp”. For that to work properly, run “ldconfig”.
Run “suricata -V” and it should report version 0.8.2.
To use Suricata in IDS mode, pass -i to the command line. Example
suricata -c /etc/suricata/suricata.yaml -i eth0
Tags: ids, Suricata, Ubuntu
Posted in Suricata, Ubuntu, ids | 2 Comments »
April 19th, 2010
Today the OISF development team released 0.8.2 of the Suricata IDS/IPS engine. I feel this is definitely the best release so far. Read the announcement here. In short, stability was improved, memory footprint reduced, performance improved and new features were added.
One of the tools we used to help improve the engine is a fuzzer created by Will Metcalf, our QA lead. In short, the script takes a pcap file, runs it through editcap (part of wireshark) altering a number of random bytes, then feeds the altered pcap file to Suricata. This resulted in many interesting corner cases. Naturally the script makes sure you don’t forget to enable “ulimit -c unlimited” and such
More on that script can be found on Will’s blog node5.
For the next period we’ll be working on resolving a number of open issues. There are still a number of improvements we need to make to the relation between our app layer decoding modules and our detection engine. Next to this we’re still missing support for a number of rule keywords, such as asn1 and http_headers. We’re also working on getting our CUDA accelaration into a more usable shape. This release improved it slightly, by making it work on x86_64, but it’s still not useful in production environments.
So as usual, enough to do! Meanwhile, we’re looking for feedback on our release!
Tags: ids, IPS, Suricata
Posted in IPS, Suricata, ids, oisf | 1 Comment »
March 31st, 2010
It’s been quite a while since I received my review copy of Magnus Mischel’s ModSecurity book titled “ModSecurity 2.5″ but I finally found the time to read it and write up my review. As the title suggest it’s a book about the ModSecurity Web Application Firewall (WAF) module for Apache and about version 2.5 of it specifically. There are some books about the 1.x series of ModSecurity. It’s great that there is a book about the 2.x ModSecurity series now as ModSecurity 2.x is very different from the 1.x series.
The ModSecurity module is very powerful but also very complex. It’s pretty trivial to add a few rules blocking some attacks, but when trying to protect large web applications such as OWA things get complicated very quickly. But even with a smaller system like WordPress I found that finding the right approach is not trivial. While there is an online manual and there are an array of blogposts (some written by me even), a good overview of ModSecurity’s features and how to really deploy it properly and effectively in complex environments is lacking. This is what I hoped to find in this book.
Giving this expectation the book slightly disappointed me. But let me start out with what I liked about the book.
The book gives a broad overview of how ModSecurity can be used. It deals with the obvious parts like compilation, installation and setting up, but also handles more interesting parts like virtual patching, performance profiling, the difference between “positive” and “negative” security approaches, REMO (a web based open source ModSecurity rule editor) and more. I learned quite a bit here, for example about directives to deal with credit card numbers.
Where it falls short is mostly in the lack of depth. It touches a lot of subjects, but most of them only pretty briefly. Next to this a view chapters could be organized a little better, especially in the first couple of chapters. I think what would really improve this book is adding the approach done by my favorite ModSecurity book so far, Ryan C. Barnetts “Preventing Web Attacks With Apache”. In that book a flawed application is introduced (Buggy Bank) and much time is spend on explaining how things are broken and where ModSecurity can and cannot help.
My verdict is that “ModSecurity 2.5″ is a good introductionary book into ModSecurity, but that it’s missing some depth to be much more than that. Being someone that has quite a bit of ModSecurity experience, including writing pretty complex rulesets, I had hoped for more help on dealing with those. But all being said, I still recommend this book to anyone that is in need of a good introduction into ModSecurity. I recommend picking up Ryan C. Barnetts “Preventing Web Attacks With Apache” book alongside with it, even though it deals with ModSecurity 1.x. I think together they will provide enough depth to deal with a real world environment.
On a final note I’d like to mention that Ivan Ristic, the original ModSecurity creator, has also written a new book on ModSecurity. I haven’t read that yet, but Ivan’s first book was excellent.
Tags: modsecurty
Posted in Books, ModSecurity | No Comments »
February 20th, 2010
One area of interest in the development of Suricata is hardware acceleration. Using the GPU is particularly interesting, as they are cheap and widely available. We’ve been looking at using the GPU to speed up pattern matching as a first step. Since OpenCL promises to be a cross platform multi vendor API for doing this we first looked at OpenCL. But we were never able to get something stable out of it, not on the NVIDIA drivers in Linux anyway. As that didn’t go anywhere we decided to use CUDA for the time being. CUDA obviously is NVIDIA only. Once we have CUDA fully running we may revisit OpenCL or look at other implementations like AMD/ATI’s stream API.
What we have so far is a implementation our 2 gram SBNDM pattern matcher algorithm in CUDA. The detection thread(s) currently send packets one by one to a central dispatcher thread that controls the GPU. This setup is far from ideal performance wise, but our first goal was to get it working at all. Currently on my desktop CUDA actually slows things down.
In the next weeks and months we plan to do some redesigning of the CUDA implementation and it’s integration into the engine. We plan to send the packets in batches to the dispatcher thread right after the decoders have determined what the payload portion of a packet is. The (separate) detection thread(s) can then process the results of the GPU when they get to a packet. By using the CUDA scanning async like this we hope that we can reduce the costs of the transfer of packets from and to the card.
Currently the code in the tree can be activated by passing the “–enable-cuda” option to ./configure. Next, in the configuration file enable the cuda pattern matcher by setting the “mpm-algo” option to “b2g_cuda”. As a first test, run the CUDA unittests (assuming you enabled the building of the unittests too) by using “suricata -uUCuda”. Please note that currently running all unittests will fail if CUDA is enabled.
The code is only tested on 32bit Linux at the moment. There are some issues with 64bit that we’re resolving right now. We’re expecting to be continuously updating this code, so be sure to work with the most current version of the git repo all the time!
Let us know your experiences!
Tags: cuda, gpu, pattern matching, Suricata
Posted in Suricata, oisf | No Comments »
February 20th, 2010
Yesterday the OISF development team released Suricata 0.8.1. This release is much improved from our December 31st release. It is way more stable, performs better and has more features. Thanks to the now included HTP library we have much better HTTP handling. The stream engine has seen massive improvements. Initial experimental CUDA code has been added. Initial Win32 support has been added. We’ve added number of missing rule keywords. Many bugs were fixed.
Personally I’m very excited about the help we have gotten from the community. Quite a few patches from community members were applied in this release. Thanks everyone!
Next week the OISF team and a number of experts are meeting up in Istanbul. We’ll be working on crunching a number of technical challenges, sharing ideas and we will start our brainstorming on future development. If you have any ideas about where you think IDS/IPS should go, please let us know so we can discuss it and possibly include it in our future plans.
Tags: cuda, htp, oisf, Suricata
Posted in IPS, Suricata, ids, oisf | No Comments »
January 4th, 2010
If you’re running into issues with Suricata, it may be worth spending some time looking at the debugging options.
To enable the debugging code, pass “–enable-debug” to configure.
./configure –enable-debug
And make & make install again. Make sure that during compilation you see -DDEBUG in the gcc commands.
Then to really enable it at runtime, pass the SC_LOG_LEVEL
SC_LOG_LEVEL=Debug
Depending on how you run the engine, this will output massive amounts of debugging info. Thats why we added a pcre regex filter option.
SC_LOG_OP_FILTER=regex
The regex currently is case sensitive. It will be matched against the full debug line. For example if you want to want to see only output related to the HTP module do something like:
SC_LOG_LEVEL=Debug SC_LOG_OP_FILTER=”htp” suricata -c suricata.yaml -r /path/to/file.pcap
Or maybe you want the stream messages as well:
SC_LOG_LEVEL=Debug SC_LOG_OP_FILTER=”(htp|stream)” suricata -c suricata.yaml -r /path/to/file.pcap
You can also control the logging format by passing the SC_LOG_FORMAT environment variable. By default it’s set to “[%i] %t – (%f:%l) <%d> (%n) — “.
The following format specifiers are available:
t timestamp
p process id (pid)
i thread id
m thread module name
d log level
f filename
l line number
n function name
Example:
SC_LOG_FORMAT=”[%i] %t – (%f:%l) <%d> (%n) — “
Putting it all together:
SC_LOG_LEVEL=Debug SC_LOG_FORMAT=”[%i] %t – (%f:%l) <%d> (%n) — ” SC_LOG_OP_FILTER=”(htp|stream)” suricata -c suricata.yaml -r /path/to/file.pcap
If you have any questions or suggestions, let me know!
Tags: debug, oisf, Suricata
Posted in Suricata, oisf | No Comments »
December 31st, 2009
Today we’ve finally released the first public version of Suricata, the Open Source IDS/IPS developed by the Open Information Security Foundation. With a team of great people we’ve been working really hard to get this ready. Please see the full announcement here.
As it’s lead developer I’m very much interested in getting feedback, bug reports and such. We run our ticket system in a redmine install at https://redmine.openinfosecfoundation.org/ If you have any feedback, please register an account and let us know what you think.
If you’re running into any issue, reconfigure and recompile the engine with –enable-unittests and –enable-debug and send us the output of “suricata -u” this will run all the unittests (1191 currently). If everything is set up properly, they should all pass. If not, please start bugging us!
Happy new year everyone!
Tags: ids, IPS, oisf, Suricata
Posted in Suricata, oisf | No Comments »
December 30th, 2009
Things here at OISF are crazy busy since we’re wrapping up our first version of the engine. Tomorrow there will be a first release! Stay tuned!
Tags: oisf, Suricata
Posted in Suricata, oisf | 1 Comment »
November 3rd, 2009
Daniele Sluijters has spend quite an effort at creating Vuurmuur rpms for Fedora 11 and CentOS 5, both 32 bit and 64 bit. The packages are available at the Vuurmuur ftp-server here: ftp://ftp.vuurmuur.org/releases/0.7/contrib/ Currently we have packages for 0.7, hopefully 0.8beta2 will follow later. Thanks Daniele!
Tags: rpm, Vuurmuur
Posted in Vuurmuur | No Comments »
Removing Trac ticket comment spam in Debian Lenny
April 23rd, 2010The Vuurmuur website runs Trac and overall I’m pretty happy with it. The only thing that Trac doesn’t do well, is dealing with spammers. Spammers target Trac a lot, so that’s a real problem.
To prevent spammers from making it through, I run Scallywhack and a number of custom ModSecurity rules. So far, spams only made it through as new tickets in the ticket tracker, so I installed the TicketDeletePlugin.
Yesterday, I saw the first spam as a comment to an existing and valid ticket. Like tickets themselves, ticket comments can not be removed by Trac by default. Luckily, upstream Trac seems to have fixed this. I’m running Debian’s version of Trac 0.11.1 however, so I decided to patch that. The patches in the Trac ticket #454 didn’t apply cleanly, so I had to patch it manually. To save others the work, it’s available here: http://www.inliniac.net/files/trac_0.11.1-debian-comment_edit.patch
To use it, make a copy of your /usr/share/pyshared/trac directory.
Next, go into the trac directory and run the command:
patch -p1 < /path/to/trac_0.11.1-debian-comment_edit.patch
After this, each comment in the comment system will have a “edit” button and you can remove the spam message content. It’s not possible to remove the entire comment, but this works for me.
Tags: comment spam, Debian, trac
Posted in Debian, ModSecurity | No Comments »