Posts Tagged ‘oisf’

« Older Entries

Suricata 0.8.1 released

Saturday, February 20th, 2010

Yesterday the OISF development team released Suricata 0.8.1. This release is much improved from our December 31st release. It is way more stable, performs better and has more features. Thanks to the now included HTP library we have much better HTTP handling. The stream engine has seen massive improvements. Initial experimental CUDA code has been added. Initial Win32 support has been added. We’ve added number of missing rule keywords. Many bugs were fixed.

Personally I’m very excited about the help we have gotten from the community. Quite a few patches from community members were applied in this release. Thanks everyone!

Next week the OISF team and a number of experts are meeting up in Istanbul. We’ll be working on crunching a number of technical challenges, sharing ideas and we will start our brainstorming on future development. If you have any ideas about where you think IDS/IPS should go, please let us know so we can discuss it and possibly include it in our future plans.

Tags: , , ,
Posted in IPS, Suricata, ids, oisf | No Comments »

Suricata debugging

Monday, January 4th, 2010

If you’re running into issues with Suricata, it may be worth spending some time looking at the debugging options.

To enable the debugging code, pass “–enable-debug” to configure.

./configure –enable-debug

And make & make install again. Make sure that during compilation you see -DDEBUG in the gcc commands.

Then to really enable it at runtime, pass the SC_LOG_LEVEL

SC_LOG_LEVEL=Debug

Depending on how you run the engine, this will output massive amounts of debugging info. Thats why we added a pcre regex filter option.

SC_LOG_OP_FILTER=regex

The regex currently is case sensitive. It will be matched against the full debug line. For example if you want to want to see only output related to the HTP module do something like:

SC_LOG_LEVEL=Debug SC_LOG_OP_FILTER=”htp” suricata -c suricata.yaml -r /path/to/file.pcap

Or maybe you want the stream messages as well:

SC_LOG_LEVEL=Debug SC_LOG_OP_FILTER=”(htp|stream)” suricata -c suricata.yaml -r /path/to/file.pcap

You can also control the logging format by passing the SC_LOG_FORMAT environment variable. By default it’s set to “[%i] %t – (%f:%l) <%d> (%n) — “.

The following format specifiers are available:

t timestamp
p process id (pid)
i thread id
m thread module name
d log level
f filename
l line number
n function name

Example:

SC_LOG_FORMAT=”[%i] %t – (%f:%l) <%d> (%n) — “

Putting it all together:

SC_LOG_LEVEL=Debug SC_LOG_FORMAT=”[%i] %t – (%f:%l) <%d> (%n) — ” SC_LOG_OP_FILTER=”(htp|stream)” suricata -c suricata.yaml -r /path/to/file.pcap

If you have any questions or suggestions, let me know!

Tags: , ,
Posted in Suricata, oisf | No Comments »

Suricata released!

Thursday, December 31st, 2009

Today we’ve finally released the first public version of Suricata, the Open Source IDS/IPS developed by the Open Information Security Foundation. With a team of great people we’ve been working really hard to get this ready. Please see the full announcement here.

As it’s lead developer I’m very much interested in getting feedback, bug reports and such. We run our ticket system in a redmine install at https://redmine.openinfosecfoundation.org/ If you have any feedback, please register an account and let us know what you think.

If you’re running into any issue, reconfigure and recompile the engine with –enable-unittests and –enable-debug and send us the output of “suricata -u” this will run all the unittests (1191 currently). If everything is set up properly, they should all pass. If not, please start bugging us!

Happy new year everyone!

Tags: , , ,
Posted in Suricata, oisf | No Comments »

First Suricata release tomorrow

Wednesday, December 30th, 2009

Things here at OISF are crazy busy since we’re wrapping up our first version of the engine. Tomorrow there will be a first release! Stay tuned!

Tags: ,
Posted in Suricata, oisf | 1 Comment »

OISF engine on ARM

Saturday, October 31st, 2009

Today I installed a Qemu virtual machine with the ARM architecture. I think ARM is becoming an interesting architecture as smartphones and many home routers use it. I was interested in seeing if our OISF engine would compile and run properly on it. So far it seems really well. Compilation was without issue, all our current 800+ unittests ran successfully and it seems to run just fine so far. Too bad the virtual machine is so slow though…

Also, since a few weeks I have a Android based smartphone. Since it runs Linux and uses an ARM CPU I think/hope it should be able to run the engine. I hope to find some time soon to look into Android development for this…

Tags: , , ,
Posted in oisf | No Comments »

OISF engine development update(2)

Wednesday, September 30th, 2009

Another quick update on the development of the OISF engine. Overall development is going great. Basics like signature keywords, stream reassembly, ip defragmentation are nearing completion. Unified1 + barnyard was already working for quite some time, but now we also have unified2 compatible output. I’ve tested this to work with barnyard2 and Sguil which works nicely.

We have the first versions of our new YAML based configuration format checked in, a brand new logging API, midstream pickup support in our Stream engine, native PFRING support and many other additions.

Next up in development is IP reputation support in the engine, support for advanced vars and more L7 modules.

The IP reputation is one of the things we have high expectation about. In working group discussions we defined 16 categories for which to keep reputation, for example: spammer, cnc, p2p, etc. We’re currently designing the datatypes for holding as much of this info in memory as possible.

The advanced vars is the idea of applying ModSecurity’s var collections to our engine. They will be like Snort’s flowbits, but then more advanced. At least 2 more types will be supported: integers and raw buffers. Integers should enable signatures to modify counters, but also compare various counters with each other.

In the L7 modules development we’re putting a big focus on HTTP (more on that later). We’re also currently working on a DCE/RPC decoder. For this work we hired Kirby Kuehl from Breaking Point. The L7 framework I’m still working on should make development of modules for new protocols fairly straightforward.

We expect to announce the name and mascot of our engine soon. Also our bylaws should also finally been done almost now, just as the consortium license. Once thats all into place, we expect to open up our code to the public quite soon. Exciting times!

Tags: , , , , ,
Posted in oisf | No Comments »

OISF engine development update

Sunday, August 16th, 2009

The last month has been crazy busy. Development of the engine is progressing nicely. My own role has been assigning tasks to our coders, guiding them, reviewing their work, integrating it and of course write code. We currently have nine people coding, not all full time though, and are still looking for more coders.

Progress has been made on a number of things: we have many more decoders, threading updates, a stats subsystem, stream tracking and reassembly, a L7 protocol parser framework and many more unittests. We’re working on OpenCL hardware accelaration, although we’re running into driver issues, so that may take some time before it’s usable.

On the QA side Will Metcalf is busy setting up an automated test rig, doing daily tests runs of our unittests on various platforms and with different compiler settings and such. When that is done pcap based tests and live traffic testing is next.

We have set up a number of “working group” mailinglists that discuss different subjects such as a configuration language and a rule language. Most are still ongoing, however the configuration language discussion seems to have come to a conclusion.

For the configuration language the discussion has settled on using YAML, a structured but still nicely editable format. It has many language bindings, so I hope management tools will be built for it later.

Other discussions, such as about the ip reputation, are still ongoing. You are very welcome to share your ideas with the group.

Like stated above, we’re still looking for coders. If you are a C coder and you’re interested in working with and for us, send us your resume!

Tags: , , ,
Posted in oisf | No Comments »

DC meeting

Tuesday, July 21st, 2009

So I just got back from Washington D.C. where we had our first public meeting for the OISF. I think it went very well as there were more people than expected. The attendees came from all parts from the industry & government. Overall reception was very positive and we’ve gotten many offers for help in development & testing.

Around the public meetings we had private meetings with a number of companies and I’m very happy that three of them commited to the project already:

Endace the New Zealand based hardware acceleration company was first to commit. They are providing us with hardware and time from their coders. Naturally they will be interested in getting our code to perform as good as possible on their hardware, but they have offered to assist in the general development of the engine as well.

Breach Security is supporting us too. They are providing us with coding time of Brian Rectanus, the current developer and maintainer of ModSecurity. Given my enthusiasm about ModSecurity, no one will be surprised that I’m really excited to having Brian and Breach involved. Naturally, they are going to help us make sure our engine excels at HTTP security. More on that in a later post.

Last (for now), we’re getting support from Nitro Security. Having worked with Nitro in the past I’m really excited about this as well, especially as Nitro has an IPS interest. Of course inliniac cares a lot about IPS! :) The form of Nitro’s support is still to be determined, but it’ll likely be in the form of time from their coders.

At this point, contributions like this (coding support, QA hardware) is what we are interested in most. We’re talking to a number of other companies for setting similar partnerships. We’ll announce them as soon as we know more.

On a last note, I’d like to thank Frank Knobbe, Daniel Peck and Richard Bejtlich for attending the meeting. It was great finally meeting you guys in person and thanks for your great input. Same goes for the other people that were there, thanks a lot for coming!

We will be publishing our meeting notes soon. Stay tuned!

Tags: , , , ,
Posted in oisf | No Comments »

OISF meeting in DC next July

Tuesday, June 30th, 2009

We’re doing a public OISF meeting in DC next July. Everyone thats interested, please show up! Here is the original announcement:

We'll be having a public forum and brainstorming session in Washington
DC on July 16th, 2009! This session will be a mix of technical and
political issues.

We encourage our current and potential consortium members, potential
users and resellers, as well as future end users to attend. We very much
want to hear from all in a discussion format what is most important to
you, and what you need to have in the next iteration of IDS. The
discussion on the lists has been great, but most often even better
things come to life when a lot of smart folks are in the same room at
the same time, as we've seen at our prior brainstorming sessions.

We'll be getting quite technical, but we'll also answer any and every
question about the politics, goals, and funding sources of the
foundation. We know this is a very strange situation we have, being
funded by DHS to create open source security software.

So please plan to attend, July 16th in Washington DC, at the SRI
Building in Rosslyn:
http://www.sri.com/contact/wdc.html

If you plan to or are rather sure you'll be there please drop an email
to Matt Jonkman, we need an approximate headcount for the
catering, provided courtesy of SRI.

If you can't make this one don't worry, we are planning similar meetings
through the development cycle on the west coast and in Europe. We want
to hear every idea we can get!

I’ll be there personally, as will (most of) the rest of the team be. Look forward to meeting everyone there!

Tags:
Posted in oisf | No Comments »

Chicago

Thursday, May 28th, 2009

Next week I’ll be in Chicago, IL for a OISF team meeting. We’ll be discussing features, work flow, job applications, contractors, etc. I’ll probably update my blog from there on the progress. If you’re interested in OISF and/or you’re around there, please let me know. Maybe we can try to meet up!

Tags:
Posted in oisf | No Comments »

« Older Entries