I’ve finally given in to the hype and got an account on Twitter. I must say that so far I’m liking it more than I expected. It seems almost everyone from the infosec community is active on the service. I am updating it nearly daily about (among other things) the OISF development I’m doing.
If you’re interested follow me here: http://twitter.com/inliniac
We’re doing a public OISF meeting in DC next July. Everyone thats interested, please show up! Here is the original announcement:
We'll be having a public forum and brainstorming session in Washington DC on July 16th, 2009! This session will be a mix of technical and political issues. We encourage our current and potential consortium members, potential users and resellers, as well as future end users to attend. We very much want to hear from all in a discussion format what is most important to you, and what you need to have in the next iteration of IDS. The discussion on the lists has been great, but most often even better things come to life when a lot of smart folks are in the same room at the same time, as we've seen at our prior brainstorming sessions. We'll be getting quite technical, but we'll also answer any and every question about the politics, goals, and funding sources of the foundation. We know this is a very strange situation we have, being funded by DHS to create open source security software. So please plan to attend, July 16th in Washington DC, at the SRI Building in Rosslyn: http://www.sri.com/contact/wdc.html If you plan to or are rather sure you'll be there please drop an email to Matt Jonkman, we need an approximate headcount for the catering, provided courtesy of SRI. If you can't make this one don't worry, we are planning similar meetings through the development cycle on the west coast and in Europe. We want to hear every idea we can get!
I’ll be there personally, as will (most of) the rest of the team be. Look forward to meeting everyone there!
Next to creating a new IDS with the OISF project I’ve been busy lately assisting Digital Bond with their Quickdraw project. The purpose of the project is to create a passive network based event logger for SCADA networks. Digital Bond has now released a first beta of the project here. Check it out!
Next week I’ll be in Chicago, IL for a OISF team meeting. We’ll be discussing features, work flow, job applications, contractors, etc. I’ll probably update my blog from there on the progress. If you’re interested in OISF and/or you’re around there, please let me know. Maybe we can try to meet up!
The OISF is a non profit foundation and we’ve created a bylaws document to govern it that is now up for comments. See the announcement here. It’s a draft so if you have comments about it, please speak up soon so we can see if it needs to be adjusted!
One thing that excites me a lot is that it also specifies the OSS license we’re going to use: the GPLv3.
Funny how things go: not long ago I posted here that I was looking for (contract) work, today I’m posting that we’re looking for people to work for us at the OISF project 
Anyway, have a look at Matt Jonkman’s announcement here.
If you’re interested or know someone that is, please contact us!
A new version of Vuurmuur is out: 0.7. This release mainly fixes bugs and build issues. Translations are generated and installed again, lots of traffic shaping fixes were made.
Support for pmtu MSS clamping was added, as was support for NAT source port randomization.
See http://www.vuurmuur.org/trac/wiki/Changelog for all changes.
Debs for Debian and Ubuntu are available, see
http://www.vuurmuur.org/trac/wiki/InstallationDebian
The source installer and Autopackage are on the ftp server:
ftp://ftp.vuurmuur.org/releases/0.7/
Looking forward, I’m planning on improving the services handling in 0.8. Especially supporting all protocols from /etc/protocols, instead of just a small list of hardcodes ones. Check http://www.vuurmuur.org/trac/milestone/0.8 to monitor the plans and progress on the 0.8 release. Suggestions & help are welcome!
The next stable version of Vuurmuur, 0.7, is getting close. Last week I released release candidate 3. If you’re a Vuurmuur user, please try 0.7rc3 and report back to me on how it works! For a list of changes, please see the closed tickets. Thanks!
I’ve been thinking about how to deal with streams in the OISF engine. We need to do stream reassembly to be able to handle spliced sessions, otherwise it would be very easy to evade detection. Snort traditionally used an approach of inspecting the packets individually and reassembling (part of) the stream in a pseudo packet, that was inspected mostly like a normal packet. Recent Snort versions, especially when Stream5 was introduced, have a so called stream api. This enables detection modules to control the reassembly better.
In Snort_inline’s Stream4 I’ve been experimenting with ways to improve stream reassembly in an inline setup. The problem with Snort’s pseudo packet scanning way of operation is that it’s after the fact scanning. Which means that any threat detected in the reassembled stream can’t be dropped anymore. The way I tried to work around this was by constantly scanning a sliding window of reassembled unacked data. It worked quite well, except for the performance of it. That was quite bad.
I’m thinking about a stream reassembler for the OISF engine that can both do the after-the-fact pseudo packet scanning and do a sliding window approach as I did in stream4inline. This would be used for the normal tcp signatures. I think it should be possible to determine the minimal size of the reassembled packet based on the signatures per port, possibly more fine grained. Of course things like target based reassembly and robust reassembly will be part of it.
In addition to this I’m thinking about a way to make modules act on the stream similary to how programs deal with sockets. Code that only wakes up if a new piece of data in that connection is received, with semantics similar to recv()/read(). I haven’t really made up my mind about how such an api should work exactly, but I think it would be very useful to detection module writers if they only have to care about handling the next chunk of data.
I haven’t implemented any of this yet, but I plan to start working on this soon. I’ll start with simple TCP state tracking that I’m planning to build on top of the flow handling already implemented. I’ll blog about this as I go…
In Januari I first wrote about my prototype code for the OISF engine. The first thing I started with when creating the code was the threading. The current code can run as a single thread or with many threads. In my normal testing I run with about 11 threads, 10 of which handle packets, 1 is a management thread.
The basic principle in the threading is that a packet is always handled by one thread at a time only. The reason for this is that it saves a lot of locking issues. If there is more than one thread, the engine can handle multiple packete simultaniously.
All functionality is created in what I call threading modules. Such a module is run in a thread. Threads can have one or more of the modules running. Examples of these modules are Decoding, Detection, Alerting, etc. I intend to make these modules plugins in the future so that 3rd party modules can be loaded without recompiling the codebase.
The threading model works both in a parallel and serial way. The parallel way can be used to have multiple threads doing the same jobs, for example have 2 threads both acquire packets, decoding, detection and alerting. The serial way of threading works differently. In that case a thread has a limited number of tasks (e.g. Decoding) and if it’s done with a packet it passes the packet on to the next thread (that for example does Detection). Both methods can be combined, which I use by default: I have 1 packet aquiring thread, 2 decoding, 2 detection, 1 verdict (I’m using nfq), and a few alerting and active response threads.
Between the (serial) threads queue’s are used to transfer the packet from one thread to another. A queue can contain multiple packets. In the above example, the nfq packet acquire thread can read packets from the queue as fast as it can and put them in it’s queue. The 2 decoding threads then get packets from this queue as fast as they can. Then they put them in the next queue where they are picked up again, etc.
Using the queue’s code paths can also be determined. It’s possible for example to have IPv4 packets be handled by different threads than IPv6 packets. Or packets with alerts differently from packets that didn’t have alerts.
One big challenge is that this is all extremely complex & configurable. Threads have to created, queue’s, queue handlers, CPU affinity can be set per thread, threading modules need to be assigned to threads, etc. I think power users & apliance builders would be interested in having all these options, but for casual users it’s probably (way) too complex to be bothered with. So some reasonable defaults need to be created, maybe in the form of having a number of pre-configure profiles.