Posts Tagged ‘Suricata’

Suricata has experimental CUDA support

Saturday, February 20th, 2010

One area of interest in the development of Suricata is hardware acceleration. Using the GPU is particularly interesting, as they are cheap and widely available. We’ve been looking at using the GPU to speed up pattern matching as a first step. Since OpenCL promises to be a cross platform multi vendor API for doing this we first looked at OpenCL. But we were never able to get something stable out of it, not on the NVIDIA drivers in Linux anyway. As that didn’t go anywhere we decided to use CUDA for the time being. CUDA obviously is NVIDIA only. Once we have CUDA fully running we may revisit OpenCL or look at other implementations like AMD/ATI’s stream API.

What we have so far is a implementation our 2 gram SBNDM pattern matcher algorithm in CUDA. The detection thread(s) currently send packets one by one to a central dispatcher thread that controls the GPU. This setup is far from ideal performance wise, but our first goal was to get it working at all. Currently on my desktop CUDA actually slows things down.

In the next weeks and months we plan to do some redesigning of the CUDA implementation and it’s integration into the engine. We plan to send the packets in batches to the dispatcher thread right after the decoders have determined what the payload portion of a packet is. The (separate) detection thread(s) can then process the results of the GPU when they get to a packet. By using the CUDA scanning async like this we hope that we can reduce the costs of the transfer of packets from and to the card.

Currently the code in the tree can be activated by passing the “–enable-cuda” option to ./configure. Next, in the configuration file enable the cuda pattern matcher by setting the “mpm-algo” option to “b2g_cuda”. As a first test, run the CUDA unittests (assuming you enabled the building of the unittests too) by using “suricata -uUCuda”. Please note that currently running all unittests will fail if CUDA is enabled.

The code is only tested on 32bit Linux at the moment. There are some issues with 64bit that we’re resolving right now. We’re expecting to be continuously updating this code, so be sure to work with the most current version of the git repo all the time!

Let us know your experiences!

Suricata 0.8.1 released

Saturday, February 20th, 2010

Yesterday the OISF development team released Suricata 0.8.1. This release is much improved from our December 31st release. It is way more stable, performs better and has more features. Thanks to the now included HTP library we have much better HTTP handling. The stream engine has seen massive improvements. Initial experimental CUDA code has been added. Initial Win32 support has been added. We’ve added number of missing rule keywords. Many bugs were fixed.

Personally I’m very excited about the help we have gotten from the community. Quite a few patches from community members were applied in this release. Thanks everyone!

Next week the OISF team and a number of experts are meeting up in Istanbul. We’ll be working on crunching a number of technical challenges, sharing ideas and we will start our brainstorming on future development. If you have any ideas about where you think IDS/IPS should go, please let us know so we can discuss it and possibly include it in our future plans.

Suricata debugging

Monday, January 4th, 2010

If you’re running into issues with Suricata, it may be worth spending some time looking at the debugging options.

To enable the debugging code, pass “–enable-debug” to configure.

./configure –enable-debug

And make & make install again. Make sure that during compilation you see -DDEBUG in the gcc commands.

Then to really enable it at runtime, pass the SC_LOG_LEVEL

SC_LOG_LEVEL=Debug

Depending on how you run the engine, this will output massive amounts of debugging info. Thats why we added a pcre regex filter option.

SC_LOG_OP_FILTER=regex

The regex currently is case sensitive. It will be matched against the full debug line. For example if you want to want to see only output related to the HTP module do something like:

SC_LOG_LEVEL=Debug SC_LOG_OP_FILTER=”htp” suricata -c suricata.yaml -r /path/to/file.pcap

Or maybe you want the stream messages as well:

SC_LOG_LEVEL=Debug SC_LOG_OP_FILTER=”(htp|stream)” suricata -c suricata.yaml -r /path/to/file.pcap

You can also control the logging format by passing the SC_LOG_FORMAT environment variable. By default it’s set to “[%i] %t – (%f:%l) <%d> (%n) — “.

The following format specifiers are available:

t timestamp
p process id (pid)
i thread id
m thread module name
d log level
f filename
l line number
n function name

Example:

SC_LOG_FORMAT=”[%i] %t – (%f:%l) <%d> (%n) — “

Putting it all together:

SC_LOG_LEVEL=Debug SC_LOG_FORMAT=”[%i] %t – (%f:%l) <%d> (%n) — ” SC_LOG_OP_FILTER=”(htp|stream)” suricata -c suricata.yaml -r /path/to/file.pcap

If you have any questions or suggestions, let me know!

Suricata released!

Thursday, December 31st, 2009

Today we’ve finally released the first public version of Suricata, the Open Source IDS/IPS developed by the Open Information Security Foundation. With a team of great people we’ve been working really hard to get this ready. Please see the full announcement here.

As it’s lead developer I’m very much interested in getting feedback, bug reports and such. We run our ticket system in a redmine install at https://redmine.openinfosecfoundation.org/ If you have any feedback, please register an account and let us know what you think.

If you’re running into any issue, reconfigure and recompile the engine with –enable-unittests and –enable-debug and send us the output of “suricata -u” this will run all the unittests (1191 currently). If everything is set up properly, they should all pass. If not, please start bugging us!

Happy new year everyone!

First Suricata release tomorrow

Wednesday, December 30th, 2009

Things here at OISF are crazy busy since we’re wrapping up our first version of the engine. Tomorrow there will be a first release! Stay tuned!