Ips on Inliniac

Suricata bits, ints and vars

Tue, 20 Dec 2016 18:37:05 +0000

Since the beginning of the project we’ve spoken about variables on multiple levels. Of course flowbits defined by the Snort language came first, but other flow based variables quickly followed: flowints for basic counting, and vars for extracting data using pcre expressions.

I’ve always thought of the pcre data extraction using substring capture as a potentially powerful feature. However the implementation was lacking. The extracted data couldn’t really be used for much.

Suricata 3.0 is out!

Wed, 27 Jan 2016 15:33:27 +0000

Today, almost 2 years after the release of Suricata 2.0, we released 3.0! This new version of Suricata improves performance, scalability, accuracy and general robustness. Next to this, it brings a lot of new features.

New features are too numerous to mention here, but I’d like to highlight a few:

netmap support: finally a high speed capture method for our FreeBSD friends, IDS and IPS
multi-tenancy: single instance, multiple detection configs
JSON stats: making it much easier to graph the stats in ELK, etc
Much improved Lua support: many more fields/protocols available, output scripts

Check the full list here in the announcement: http://suricata-ids.org/2016/01/27/suricata-3-0-available/

New Suricata release model

Tue, 24 Nov 2015 15:54:35 +0000

As the team is back from a very successful week in Barcelona, I’d like to take a moment on what we discussed and decided on with regards to development.

One thing no one was happy with is how the release schedules are working. Releases were meant to reasonably frequent, but the time between major releases was growing longer and longer. The 2.0 branch for example, is closing in on 2 years as the stable branch. The result is that many people are missing out on many of the improvements we’ve been doing. Currently many people using Suricata actually use a beta version, of even our git master, in production!

Suricata has been added to Debian Backports

Thu, 08 Jan 2015 00:34:50 +0000

Thanks to the hard work of Arturo Borrero Gonzalez, Suricata has just been added to the Debian ‘backports’ repository. This allows users of Debian stable to run up to date versions of Suricata.

The ‘Backports’ repository makes the Suricata and libhtp packages from Debian Testing available to ‘stable’ users. As ’testing’ is currently in a freeze, it may take a bit of time before 2.0.5 and libhtp 0.5.16 appear.

Anyway, here is how to use it.

Profiling Suricata with JEMALLOC

Tue, 23 Dec 2014 15:34:23 +0000

JEMALLOC is a memory allocation library: http://www.canonware.com/jemalloc/

It offers many interesting things for a tool like Suricata. Ken Steele of EZchip (formerly Tilera) made me aware of it. In Ken’s testing it helps performance.

Install

wget http://www.canonware.com/download/jemalloc/jemalloc-3.6.0.tar.bz2
tar xvfj jemalloc-3.6.0.tar.bz2
cd jemalloc-3.6.0
./configure --prefix=/opt/jemalloc/
make
sudo make install

Then use it by preloading it:

LD_PRELOAD=/opt/jemalloc/lib/libjemalloc.so ./src/suricata -c suricata.yaml -l tmp/ -r ~/sync/pcap/sandnet.pcap -S emerging-all.rules -v

I haven’t benchmarked this, but if you’re running a high performance setup it may certainly be worth a shot.

Crossing the Streams in Suricata

Sat, 20 Dec 2014 23:27:24 +0000

At it’s core, Suricata is a packet processor. It reads packets and pushes them through a configurable pipeline. The 2nd most important processing unit in Suricata is the flow. In Suricata we use the term flow for the bidirectional flows of packets with the same 5 tuple (proto, src ip, dst ip, sp, dp. Vlans can be added as well). In fact, much of Suricata’s threading effort revolves around the flow. In the 2 main runmodes, autofp and workers, flow based load balancing makes sure that a all packets of a single flow always go through the same threading pipeline. In workers this means one single thread, in autofp 2: the capture thread and a stream/detect/output thread.

SMTP file extraction in Suricata

Tue, 11 Nov 2014 10:47:42 +0000

In 2.1beta2 the long awaited SMTP file extraction support for Suricata finally appeared. It has been a long development cycle. Originally started by BAE Systems, it was picked up by Tom Decanio of FireEye Forensics Group (formerly nPulse Technologies) followed by a last round of changes from my side. But it’s here now.

It contains:

a MIME decoder
updates to the SMTP parser to use the MIME decoder for extracting files
SMTP JSON log, integrated with EVE
SMTP message URL extraction and logging

As it uses the Suricata file handling API, it shares almost everything with the existing file handling for HTTP. The rule keyword work and the various logs work automatically with SMTP as well.

Suricata Flow Logging

Mon, 28 Jul 2014 19:09:07 +0000

Pretty much from the start of the project, Suricata has been able to track flows. In Suricata the term ‘flow’ means the bidirectional flow of packets with the same 5 tuple. Or 7 tuple when vlan tags are counted as well.

Such a flow is created when the first packet comes in and is stored in the flow hash. Each new packet does a hash look-up and attaches the flow to the packet. Through the packet’s flow reference we can access all that is stored in the flow: TCP session, flowbits, app layer state data, protocol info, etc.

More on Suricata lua flowints

Tue, 23 Apr 2013 10:17:52 +0000

This morning I added flowint lua functions for incrementing and decrementing flowints. From the commit:

Add flowint lua functions for incrementing and decrementing flowints.

First use creates the var and inits to 0. So a call:
    a = ScFlowintIncr(0)
Results in a == 1.

If the var reached UINT_MAX (2^32), it’s not further incremented. If the var reaches 0 it’s not decremented further.

Calling ScFlowintDecr on a uninitialized var will init it to 0.

Suricata Lua scripting flowint access

Mon, 22 Apr 2013 16:16:30 +0000

A few days ago I wrote about my Emerging Threats sponsored work to support flowvars from Lua scripts in Suricata.

Today, I updated that support. Flowvar ‘sets’ are now real time. This was needed to fix some issues where a script was invoked multiple times in single rule, which can happen with some buffers, like HTTP headers.

Also, I implemented flowint support. Flowints in Suricata are integers stored in the flow context.

Suricata: Handling of multiple different SYN/ACKs

Fri, 19 Apr 2013 07:53:00 +0000

When processing the TCP 3 way handshake (3whs), Suricata’s TCP stream engine will closely follow the setup of a TCP connection to make sure the rest of the session can be tracked and reassembled properly. Retransmissions of SYN/ACKs are silently accepted, unless they are different somehow. If the SEQ or ACK values are different they are considered wrong and events are set. The stream events rules will match on this.

Suricata 1.4 is out

Thu, 13 Dec 2012 17:54:22 +0000

About 5 months after 1.3 came out we’ve released 1.4, and we’ve been quite busy. Eric Leblond’s post here has all the stats and graphs. There are three big new features: unix socket, ip reputation and luajit. For each of these the same is true: it’s usesable now, but it’s the potential that we’re most excited about. Over the next months we’ll be extending each of those to be even more useful. We’re very much interested in ideas and feedback.

IPv6 Evasions, Scanners and the importance of staying current

Tue, 11 Dec 2012 16:13:49 +0000

Lots of activity on the IPv6 front lately. There was a talk on a conference on bypassing IDS using IPv6 tricks. Also a new scan tool (Topera) claimed to scan a host while staying below the radar of an IDS was released. To start with the latter, even though Suricata doesn’t have a dedicated port scan detector, the tool’s traffic lights up like a Christmas tree. The trick it pulls is to pack a lot of duplicate DST OPTS extension headers in the IPv6 packets. These options are just fillers, the only options they use are the “pad” option. In Suricata we’ve had an event for duplicate DST OPTS headers since 1.3 and the padding only headers generate an event in 1.4. Both alerts will be very noisy, so calling this a stealth attack rather dubious.

Closing in on Suricata 1.4

Thu, 29 Nov 2012 16:50:15 +0000

I just made Suricata 1.4rc1 available with some pretty exciting features: unix socket mode and IP reputation.

Unix socket

First of all, Eric Leblond’s work on the Unix socket was merged. The unix socket work consists of two parts. The unix socket protocol implementation and a new runmode.

The protocol implementation is based on JSON messages over unix socket. Eric will be fully documenting it soon. Currently the commands are limited to shutting down and getting some basic stats. This part isn’t very exciting yet, but the groundwork for many future extensions has been laid.

IP Reputation in Suricata

Wed, 21 Nov 2012 19:22:01 +0000

Disclaimer: this work was sponsored by Emerging Threats Pro.

One thing we’ve been talking about for many years at OISF is IP Reputation. The basic idea is that many organizations have information about specific IP-addresses. This information may be that a host is infected, acts as a spam relay or many other things. We’ve always thought it might be useful to apply this info to the IDS directly.

In the last weeks I’ve developed code to load IP reputation information into Suricata. This code is now part of the Suricata git master, so it’s available to all.

Important Suricata update

Thu, 01 Nov 2012 18:16:51 +0000

We just released Suricata 1.3.3 which contains some important accuracy fixes. Also, it should be much more robust against out of memory conditions.

For those of you running Suricata in IPS mode, this is important as well. We found that rules that have the drop or reject actions, were not playing well with thresholding.

So upgrading is highly recommended!

Code changes are not too big, largest changes are due to some extra unittests:

Setting up an IPS with Fedora 17, Suricata and Vuurmuur

Sat, 13 Oct 2012 11:07:19 +0000

I recently found out that Fedora includes Vuurmuur in it’s repositories. Since Suricata is also included, I figured I would do a quick write up on how to setup a Fedora IPS. While writing it turned more into a real “howto”, so I decided to submit it to Howtoforge.

It can be found here one HowtoForge.

Vuurmuur on Fedora is at the 0.7 version, which is still the current stable. It’s rather old though, and it reminds me again I need to make sure the 0.8 branch gets to a stable release soon. The Suricata included in Fedora 17 is 1.2.1, with 1.3.2 expected to land any day now.

Suricata MD5 blacklisting

Sat, 09 Jun 2012 08:52:22 +0000

For a few months Suricata has been able to calculate the MD5 checksum of files it sees in HTTP streams. Regardless of extraction to disk, the MD5 could be calculated and logged. Martin Holste created a set of very cool scripts to use the logged MD5 to look it up at VirusTotal and some other similar services. This is done outside of Suricata. One thing I have been wanting to try is matching against these MD5’s in Suricata itself.

F-Secure AV updates and Suricata IPS

Wed, 07 Mar 2012 07:28:38 +0000

My ISP recently started providing 3 F-Secure AV copies to each of their customers. I installed it but noticed that updates timed out.

It turned out that Suricata, which runs in IPS mode, blocked the update. There were 3 Emerging Threats rules that alerted:

[1:2003614:4] ET VIRUS WinUpack Modified PE Header Inbound [1:2009557:2] ET TROJAN Yoda’s Protector Packed Binary [1:2012086:2] ET SHELLCODE Possible Call with No Offset TCP Shellcode

It seems that F-Secure uses some form of packed binaries for their updates that is often used by malware.

Suricata 1.1.1 released

Wed, 07 Dec 2011 18:34:50 +0000

A maintenance update for the Suricata 1.1 series was just released. It fixed an important issue. In some cases Suricata could crash on SMTP traffic.

The full announcement for the 1.1.1 release is here.

Naturally, the issue has also been fixed in the 1.2 development branch.

File extraction in Suricata

Tue, 29 Nov 2011 16:27:27 +0000

Today I pushed out a new feature in Suricata I’m very excited about. It has been long in the making and with over 6000 new lines of code it’s a significant effort. It’s available in the current git master. I’d consider it alpha quality, so handle with care.

So what is this all about? Simply put, we can now extract files from HTTP streams in Suricata. Both uploads and downloads. Fully controlled by the rule language. But thats not all. I’ve added a touch of magic. By utilizing libmagic (this powers the “file” command), we know the file type of files as well. Lots of interesting stuff that can be done there.

Suricata 1.1 released, 1.2 on the horizon

Thu, 10 Nov 2011 16:51:52 +0000

Today we released Suricata 1.1. This ends a rather long development cycle of more than a year. And it shows. Performance, accuracy and features were all greatly improved. I think it’s the best Suricata so far. If you’ve been looking at trying Suricata, now might be a good time to jump in.

The long development cycles should be something of the past. At our last brainstorm session, at RAID 2011, we decided to change our release policy. The aim of this policy is to do time based releases, roughly a “stable” every 2 months and a beta every other month. This way we’ll be making it much easier for users to stay current without have to run our “git master”.

RAID 2011 Thoughts

Sat, 24 Sep 2011 16:09:24 +0000

The last few days I’ve been at the Recent Advances in Intrusion Detection (RAID) conference in California. Overall it has been a very pleasant and interesting experience. The nice California weather was certainly helping a lot!

I’ve seen all talks and some were very interesting. However, being a Suricata IDS developer, I was not just interested in research for the hell of it, but I was actively scouting for ideas we could implement into Suricata. In this respect the conference was highly disappointing. Although with some of the talks I thought the idea was applicable in general security, like Erik Bosmans high speed memory tainting detection, I found nothing like that for NIDS.

Suricata IPS improvements

Mon, 31 Jan 2011 20:51:25 +0000

January has been a productive month for Suricata, especially for the IPS part of it. I’ve quite some time on adding support to the stream engine to operate differently when running inline. This was needed as dropping attacks found in the reassembled stream or the application layer was not reliable. Up until now the stream engine would offer the reassembled stream to the detection engine as soon as it was ACK’d. This meant that by definition the packets containing the data had already passed the IPS device. Simply switching to sending un-ACK’d data to the detection engine would have it’s own set of issues.

Suricata 1.1 beta 1 released

Tue, 21 Dec 2010 17:56:32 +0000

Today we’ve released Suricata 1.1 beta 1, the first beta of the upcoming Suricata 1.1 release. The official release announcement is here on the OISF website.

The main focus of the new release has been to improve performance and to add support to the features the new ET/ETpro ruleset needs. ET and ETpro have rulesets specially tuned and geared for Suricata. We’re still missing some new rule keywords that are used by VRT, so in the 1.1 beta 2 release we’ll address that.

Suricata 1.0.2 released

Thu, 02 Sep 2010 17:36:38 +0000

After some well deserved vacation I’m getting back up to speed in Suricata development. Luckily most of our dev team continued to work in my absence, making today’s 1.0.2 release possible.

The main focus of this release was fixing the TCP stream engine. Judy Novak found a number of ways to evade detection. See her blog post describing the issues.

The biggest other change is the addition of a new application layer module. The SSH parser parses SSH sessions and stops detection/inspection of the stream after the encrypted part of the session has started. So this is mainly a module focused on reducing the number of packets that need inspection, just like the SSL and TLS modules.

Setting up Suricata 0.9.0 for initial use on Ubuntu Lucid 10.04

Mon, 10 May 2010 14:27:25 +0000

The last few days I blogged about compiling Suricata in IDS and IPS mode. Today I’ll write about how to set it up for first use.

Starting with Suricata 0.9.0 the engine can run as an unprivileged user. For this create a new user called “suricata”.

useradd –no-create-home –shell /bin/false –user-group –comment “Suricata IDP account” suricata

This command will create a user and group called “suricata”. It will be unable to login as the shell is set to /bin/false.

Compiling Suricata 0.9.0 in Ubuntu Lucid 10.04 in IPS (inline) mode

Fri, 07 May 2010 08:30:30 +0000

Note: the difference with the 0.8.2 post is that addition of libcap-ng-dev. This allows Suricata to run as an unprivileged user.

Here is how to compile Suricata 0.9.0 in inline mode on Ubuntu Lucid 10.04.

First, make sure you have the “universe” repository enabled. Go to the System menu, Administration, Software Sources. There enable “Community-maintained Open Source Software (universe)”. If you’re not running a gui, edit /etc/apt/sources.list and enable the universe repository there. Don’t forget doing an “apt-get update”.

Suricata 0.9.0 released

Fri, 07 May 2010 08:08:45 +0000

Yesterday we released we first release candidate for our upcoming 1.0 release of Suricata. See the announcement on the OISF site here.

Most notable changes are the following new features:

- Support for the http_headers keyword was added

- libhtp was updated to version 0.2.3

- Privilege dropping using libcap-ng is now supported

- Proper support for “pass” rules was added

- Inline mode for Windows was added

Go get the release here: http://www.openinfosecfoundation.org/download/suricata-0.9.0.tar.gz

Compiling Suricata 0.8.2 in Ubuntu Lucid 10.04 in IPS (inline) mode

Sat, 01 May 2010 19:45:12 +0000

Yesterday I wrote about how to compile and install Suricata 0.8.2 as an IDS on Ubuntu Lucid 10.04, today I’ll explain the steps to compile and install it as an IPS. In IPS mode the engine runs in inline mode. This means that it gets it’s packets from netfilter and sets a verdict on them after inspecting them. This way we can drop packets that trigger the rules.

Suricata 0.8.2 released

Mon, 19 Apr 2010 20:38:28 +0000

Today the OISF development team released 0.8.2 of the Suricata IDS/IPS engine. I feel this is definitely the best release so far. Read the announcement here. In short, stability was improved, memory footprint reduced, performance improved and new features were added.

One of the tools we used to help improve the engine is a fuzzer created by Will Metcalf, our QA lead. In short, the script takes a pcap file, runs it through editcap (part of wireshark) altering a number of random bytes, then feeds the altered pcap file to Suricata. This resulted in many interesting corner cases. Naturally the script makes sure you don’t forget to enable “ulimit -c unlimited” and such :) More on that script can be found on Will’s blog node5.

Suricata 0.8.1 released

Sat, 20 Feb 2010 16:14:41 +0000

Yesterday the OISF development team released Suricata 0.8.1. This release is much improved from our December 31st release. It is way more stable, performs better and has more features. Thanks to the now included HTP library we have much better HTTP handling. The stream engine has seen massive improvements. Initial experimental CUDA code has been added. Initial Win32 support has been added. We’ve added number of missing rule keywords. Many bugs were fixed.

OISF IDS/IPS engine prototype intro

Wed, 07 Jan 2009 11:24:07 +0000

For over a year I’ve been working on a prototype implementation of a new IDS/IPS engine for the Open Infosec Foundation. This is not necessarily going to be the engine we’ll be using in OISF, although it’s likely that at least some of the code will be used. Discussions about features for the engine are still ongoing ( wiki, list), once that settles down we’ll see whats usable and whats not. In the worst case I still think many parts like hashing functions, pattern matcher implementations, protocol decoders, etc can be used.

Available for contract work

Mon, 05 Jan 2009 13:26:06 +0000

This year there will be a lot of work that needs to be done for the Open Infosec Foundation. And like I wrote a few days ago, a lot of work is already being done. However, most of it is unpaid at this time as it will be some months before our funding comes in. So at least until then I’m available and looking for contract work.

For the last two years I’ve been doing work as a contractor in the (open source) security field. My experience is mostly in coding in C and Perl, primarily on Snort and Snort_inline. Recently I created the (Perl language) SidReporter program for Emerging Threats. Areas I worked in: IPv6 IDS/IPS coding, signature writing, Web Application Firewalls, threading, bandwidth accounting, and more…

Looking forward to 2009: Open Infosec Foundation

Mon, 29 Dec 2008 12:15:20 +0000

The year 2008 was an exciting year to me. The biggest thing going on the infosec side was the formation of the Open Infosec Foundation. We’ve been working on it behind the scenes for more than a year now, and it’s cool that we’ve finally announced our plans. Of course, the work is just getting started. Next year, we expect to finalize our foundation setup. We’re working with the Software Freedom Law Center for setting up the foundation charter and consortium rules. While the US government is funding us initially, we hope the consortium will guarantee our long term funding. We are talking to some interesting companies already, both big and small.

Open Infosec Foundation founded!

Fri, 17 Oct 2008 22:07:59 +0000

Last week Matt Jonkman announced the formation of the Open Infosec Foundation. This foundation has been grant funded to create a new open source IDS/IPS engine. Together with Will Metcalf and of course Matt himself, I will be working on this. We want this to be a real community effort where there is a role for everyone in the infosec community. Developers, admins, vendors, goverments, research, education, everyone. There is a lot of work ahead, but that should be great fun and very inspiring. So far things are interesting already. The discussion mailinglist is growing rapidly with many ppl from the community and industry. A #oisf IRC channel was created today on freenode. Join us there to participate in discussion about this project!

Snort_inline and out of order packets

Mon, 30 Jul 2007 21:22:56 +0000

In Snort_inline’s stream4 modifications, one of the changes is that out of order TCP packets are treated differently from unmodified stream4. This can cause some new alerts to appear and some unexpected behaviour. So I’ll try to explain what happens here.

First of all let me explain quickly what out of order packets are. To put it simple, TCP packets are send out by the source host in a specific order but can arrive in a different order at the destination. Packetloss, link saturation, routing issues are among many things that can cause this. A Snort_inline specific issue is that when Snort_inline can’t keep up with the packets it needs to process, it will drop packets which causes packetloss. These packets will then have to be resent by the sending host.

Update on using realtime blacklists with ModSecurity

Thu, 01 Mar 2007 08:04:55 +0000

A few days ago I posted a blog article about stopping comment spam with ModSecurity using realtime blacklists (rbl). While the approach was working, I noted having problems with rules when I tried to match on POST methods in HTTP requests.

Luckily, ModSecurity creator Ivan Ristic was quick to point out where the problem is. I’m using the Core Ruleset for ModSecurity, and one thing that ruleset does is use the ’lowercase’ transformation. This converts all text from arguments to lowercase, so my ^POST$ match would never be able to match. So like Ivan suggested, using ^post$ solved this part.

Blocking comment spam using ModSecurity and realtime blacklists

Thu, 22 Feb 2007 22:25:45 +0000

Spammers are known to use compromised hosts from all over the world to send their messages. Many people are blocking or scoring email spam based on realtime blacklist (rbl), which contain ipaddresses of these known bad hosts. In my experience this works fairly well for email. A while ago I noticed in the ModSecurity documentation for version 2.0 that ModSecurity features an operator called rbl, that can be used to check the ipaddress of a visitor with a rbl. So I decided to see if I could use the realtime blacklists to prevent comment spam on my blog. Turns out this works great! In this post I’ll show how to get it working.

Snort_inline in svn updated to 2.6.1.3

Thu, 22 Feb 2007 07:59:05 +0000

This week SourceFire published a security advisory for (among others) Snort version 2.6.1.2, on which Snort_inline is based. So I took some time to update Snort_inline. Normally this would have taken Will and me quite some time, but since we switched to using svn those days are gone. I was able to update it in under a hour. I was very happy I blogged about the procedure to follow, since I had already forgotten about it ;-)

Snort_inline 2.6.1.2 BETA 1 released!

Tue, 23 Jan 2007 15:52:00 +0000

William Metcalf has finally released the new Snort_inline version we have been working on so hard, the first release of our code against Snort 2.6. The last release was in June 2006.

Of course, we continue to lag behind SourceFire, as they just released 2.7.0 BETA 1, but I have good hope that we will be able to keep up a little bit better the following time!

Anyway, get the release from the SourceForge download section!

Snort_inline patch updated to 2.6.1.2

Wed, 17 Jan 2007 11:55:34 +0000

With the recent Snort vulnerabilities we had to make a choice if we would backport the fixes to our Snort_inline 2.6.0.2 patch or that we would upgrade to 2.6.1.2. Upgrading makes most sense since SourceFire improves Snort with every release, but since the upgrade process has been very painful the last couple of releases, we weren’t really looking forward to it.

Earlier I wrote about my testing with Subversion for Snort_inline, and I found out that using Subversion made the upgrade procedure much easier and much less time consuming. So upgrading it was. Generally there were little changes to the Snort_inline patch required.

Setting up Subversion for Snort_inline

Wed, 17 Jan 2007 11:02:31 +0000

A reason for the slow development of Snort_inline is that we still weren’t using a version control system. Being sick of this, I decided to setup a private Subversion server to see how we could best use it. One thing that complicates the use of such a system is the fact that we maintain a patch on top of source code not maintained by ourselves. So the system must be able to deal with upstream sourcecode updates.

Detecting and blocking Phishing with Snort and ClamAV

Sun, 12 Nov 2006 18:12:31 +0000

ClamAV is a great Open Source virusscanner that can be used for detecting virusses from Snort or Snort_inline using the ClamAV preprocessor. However, by using the anti-phishing and anti-scam signatures by SaneSecurity, this combination can also be used to detect and block phishing and scam attempts. Here is how to set it up.

I’ve decided to run this on my gateway, which is a slow machine. Because I don’t want all my traffic to slow down to much, I’m not going to run the ClamAV defs, only the anti-phishing ones. The default location of the defs on my Debian Sarge system is /var/lib/clamav, so I’ve created a new directory called ‘/var/lib/clamav-phish’. Next I’ve downloaded the defs from SaneSecurity. After unzipping them and the defs were ready.

Update on Snort_inline 2.6.0.2 development

Fri, 10 Nov 2006 11:54:11 +0000

I have spend the last week trying to find a very annoying bug that caused Snort_inline to go into 100% CPU on certain traffic. It kept working, only my P3 500Mhz home gateway slowed down to between 2kb/s and 25kb/s, while normally it handles the full 325kb/s for my DSL line at around 25% CPU.

Snort comes with a number of performance measurement options. In 2.6 –enable-perfprofiling was introduced. Also, –enable-profile builds Snort for use with gprof. Next to those you can use strace and ltrace with the -c option to see the ammount of time spend in the several functions.

I already knew the problem was related to my new Stream4 code, since running Snort_inline without the ‘stream4inline’ option made the problem go away. So my performance debugging and code reviews were focussed on that code. However, the performance statistics showed no functions that took large ammounts of time in Stream4.

Rules for reported Tikiwiki vulnerabilities

Thu, 02 Nov 2006 11:02:52 +0000

Yesterday there was a mail to the bugtraq mailinglist about two types of vulnerabilties in Tikiwiki 1.9.5. The most serious is a claimed MySQL password disclosure through a special URI. The second is an XSS, also through an special URI. The message can be found here.

I wrote ‘claimed password disclosure’, because on the Tikiwiki server I run, I could not reproduce it. With that I mean the password disclosure, since I do see that Tikiwiki gives an error that reveals other information, most notably the location of the website on the local filesystem.

Snort_inline: getting closer to 2.6.0.2

Sun, 29 Oct 2006 22:40:29 +0000

I’m back from my vacation which was very nice. Hardly did any geek stuff, other than meeting up with Philippe, who lives in Paris. It was the first time I met someone I got to know through the Vuurmuur project :)

So with Snort_inline things aren’t moving as fast as I hoped, but there is certainly progress. I’m currently hunting for a few bugs. First of all I’ve seen it segfault on me once. Sadly I had forgotten to enable coredumps, so no clue as of why. Second, William and I have been ironing out some issues where the new stream4 mode was getting mixed up with the old. I think these are pretty much taken care of now. Third, there is a bug where an unified alert fired by http_inspect doesn’t contain a payload. Finally, i’m hunting what appears to be a heisenbug in the new stream reassembly, because I’ve never encountered it since I’m actually looking for it.

Snort_inline: running Snort_inline 2.6.0.2

Thu, 05 Oct 2006 08:13:29 +0000

No, it’s not released. But it wil be soon… really!

William has done most of the hard work of porting our Snort_inline patch from 2.4.5 to 2.6. I have mostly been working on improving the stream4inline modification. I have written about this before. Like the stream4inline modification in Snort_inline 2.4.5 it scans the stream in a sliding window, making it possible to drop an attack detected in the reassembled stream. The new code does the same but is much faster, at the cost of higher memory usage.

ModSecurity: rule for latest Tikiwiki vulnerability

Wed, 06 Sep 2006 13:02:57 +0000

A few days ago a new vulnerability was reported in Tikiwiki 1.9.x, the software I use for the Vuurmuur Wiki. Luckily, the Snort.org Community rules quickly had a rule for detecting the attack. Because I also run ModSecurity on the webserver, i wanted to have protection there as well. This rule should block the attack:

SecFilterSelective POST_PAYLOAD “jhot.php” “log,deny,status:403,msg:‘LOCAL tikiwiki jhot.php attempt’”

Let’s see if I ever get a hit on it. An update for Tikiwiki as been released, so that should fix the issue completely.

ModSecurity: rules against comment spam

Wed, 23 Aug 2006 08:07:40 +0000

Lately the wiki of my Vuurmuur project has been receiving quite a lot of comment spam. Although removing the spam manually is boring work, i still don’t really mind the spam, because it enables me to practice with ModSecurity rules to fight it off. So far, the spam seems to be following a pattern, in which the spam is posted by bots, and has the same general layout for longer periods of time. That makes it worthwhile to spend time on creating rules against it. Yesterday a new type of spam emerged on the wiki. The following audit_log is for one of them. I had to slightly edit it for layout reasons.

ModSecurity: more security by obscurity

Thu, 17 Aug 2006 07:27:13 +0000

Yesterday, Philippe Baumgart showed me that my obscurity setup is not yet perfect. In fact, he could very easily enter an URL that didn’t exist and caused the webserver behind my proxy to respond with a 404. In this 404 the name and the version of the webserver were exposed.

After some testing i found that adding the following to my config worked very well.

# enable output scanning in Mod Security. SecFilterScanOutput On

ModSecurity: redirection

Tue, 08 Aug 2006 22:09:50 +0000

Another nice feature of ModSecurity is rule based redirection. Lets say i want to block visitors of my website from opening the login page of wordpress, /blog/wp-login.php. I could of course just deny access to it, so the visitor gets a 403 error. This works fine, however sometimes you might want to use a more userfriendly message, for example: ‘Due to maintainance logins are currently disabled’.

To do this i first created a very simple html file called nologin.html, and placed it in the webroot of the server. Then i added the following rules to Mod_Security:

ModSecurity: directory hiding a.k.a. security by obscurity

Sun, 06 Aug 2006 20:24:07 +0000

Ok, that’s a bit misleading, because i’m not just hiding, but also blocking and logging. What i wanted is this: I’m running awstats on my reverse proxy, but i don’t want anyone to know. So i just made the entire ‘cgi-bin’ part forbidden for everyone, so that covers the script. The fact that my webserver has a cgi-bin directory is nothing special and won’t tell you i’m using awstats. However, awstats also uses icons, and these are by default in /awstats-icon/

ModSecurity: my first rules

Tue, 11 Jul 2006 09:37:33 +0000

I have been using ModSecurity for quite some time now to protect a webserver, but i never felt the need to write my own rules. Recently though, my site got quite a lot of spam in the comments of my TikiWiki based site. Since i was not willing to disable the anonymous comment posting i decided to see if i could use Mod_Security to block the spam.

The spam all looked alike with the following characteristics. It all contained a list uri’s with a pipe | before them. So decided to try the most easy way, by blocking all posts with this characteristic.

Snort_inline: idea for an improved bait-and-switch

Mon, 10 Jul 2006 22:13:09 +0000

William Metcalf recently wrote a bait-and-switch plugin for Snort_inline. The idea is that when a rule matches on certain traffic this plugin loads an iptables rule into the system that redirects the offending host to another server. This can present the user an error message such as “Access Denied” for example, but this server can also have al kinds of sniffing tools, or even be a honeypot.

As the plugin currently creates an iptables rule it only works with linux. Also, it has some difficulty with existing iptables rulesets that might be maintained by other programs, such as my own Vuurmuur. My idea is to investigate whether or not it is possible to simply do the redirection in Snort_inline itself. By rewriting the ipaddress in the IP header, it might work as well. Naturally, this would need to be done for every packet, but with a connection to either the flow engine or the stream engine, this should be able to work… just a thought…

Books: Preventing Webattacks with Apache

Mon, 10 Jul 2006 21:54:51 +0000

I just finished Preventing Webattacks with Apache by Ryan C. Barnett. Even though the title says it is about Apache it is really mostly about Mod_Security, and this is why i really love the book.

Especially cool is the part of the book where the author challenges the user to setup his ‘Buggy Bank’ example application to play with the vulnarebilities and with the possible counter measures.

This book got me even more exited about Mod_Security, which I use already to protect one webserver. I plan to check out setting up a reverse filtering web proxy soon.

Snort_inline: Adapting the TCP stream reassembler

Mon, 10 Jul 2006 16:34:25 +0000

Currently I am rewriting a modification of the TCP reassembler in Snort_inline. Snort’s TCP reassembler is called Stream4 and it works fairly well in IDS mode, however it has some serious issues in inline mode. The biggest and most important issue is that Snort_inline cannot block an attack if it is detected in the reassembled stream. In Snort_inline 2.4 we made our first attempt to fix this with the stream4inline modification.