Tcp on Inliniac

Suricata: Handling of multiple different SYN/ACKs

Fri, 19 Apr 2013 07:53:00 +0000

When processing the TCP 3 way handshake (3whs), Suricata’s TCP stream engine will closely follow the setup of a TCP connection to make sure the rest of the session can be tracked and reassembled properly. Retransmissions of SYN/ACKs are silently accepted, unless they are different somehow. If the SEQ or ACK values are different they are considered wrong and events are set. The stream events rules will match on this.

Suricata IPS improvements

Mon, 31 Jan 2011 20:51:25 +0000

January has been a productive month for Suricata, especially for the IPS part of it. I’ve quite some time on adding support to the stream engine to operate differently when running inline. This was needed as dropping attacks found in the reassembled stream or the application layer was not reliable. Up until now the stream engine would offer the reassembled stream to the detection engine as soon as it was ACK’d. This meant that by definition the packets containing the data had already passed the IPS device. Simply switching to sending un-ACK’d data to the detection engine would have it’s own set of issues.

Suricata 1.0.2 released

Thu, 02 Sep 2010 17:36:38 +0000

After some well deserved vacation I’m getting back up to speed in Suricata development. Luckily most of our dev team continued to work in my absence, making today’s 1.0.2 release possible.

The main focus of this release was fixing the TCP stream engine. Judy Novak found a number of ways to evade detection. See her blog post describing the issues.

The biggest other change is the addition of a new application layer module. The SSH parser parses SSH sessions and stops detection/inspection of the stream after the encrypted part of the session has started. So this is mainly a module focused on reducing the number of packets that need inspection, just like the SSL and TLS modules.

OISF engine prototype: streams handling

Tue, 31 Mar 2009 15:36:27 +0000

I’ve been thinking about how to deal with streams in the OISF engine. We need to do stream reassembly to be able to handle spliced sessions, otherwise it would be very easy to evade detection. Snort traditionally used an approach of inspecting the packets individually and reassembling (part of) the stream in a pseudo packet, that was inspected mostly like a normal packet. Recent Snort versions, especially when Stream5 was introduced, have a so called stream api. This enables detection modules to control the reassembly better.

Improving Snort_inline's NFQ performance

Wed, 23 Jan 2008 15:27:29 +0000

When using Snort_inline with NFQ support, it’s likely that at some point you’ve seen messages like these on the console: packet recv contents failure: No buffer space available. When the messages are appearing Snort_inline slows down significantly. I’ve been trying to find out why.

There are a number of setting that influence NFQ performance. One of them is the NFQ queue maximum length. This is a value in packets. Snort_inline takes an argument to modify the buffer length: –queue-maxlen 5000 (note: there are two dashes before queue-maxlen).

New Snort_inline TCP window normalization code in SVN

Sat, 17 Nov 2007 13:55:38 +0000

A while ago I wrote about why the TCP window scaling normalization in Snort_inline was broken by design. I also wrote about a new solution I was working on and testing that would be uploaded to SVN soon. I just committed the patch to SVN. What it does is add two new options to stream4:

norm_window: normalize the TCP window (disabled by default). This is to protect Snort_inline from being forced to queue too many packets. max_win_size: maximum size of the scaled TCP window. Packets increasing the window beyond the limit are modified.

Window scaling normalization in Snort_inline broken by design

Tue, 04 Sep 2007 15:51:25 +0000

After debugging some connection problems I found that the wscale normalization concept is flawed. I’ll describe here what is wrong with it and then move on to suggest a different solution I’m currently testing. The problem I was seeing is that some connections to some webservers stalled without an apparent reason.

First a quick reminder of why I originally came up with the wscale normalization. Stream4 originally doesn’t look at the window scaling value when determining the TCP window. This causes it to be wrong about the TCP window in about every connection, which is one of the reasons out of window packets are not dropped (this is actually a gaping evasion hole since these packets are not used in stream reassembly). This is why I decided to add window scaling support to the stream4inline extension. This works great and allows the admin to drop out of window packets. There is a problem associated with it though. The maximal window that is possible with wscaling is 1GB. This would mean that Snort_inline would in the worst case have to queue almost 1GB of data in it’s buffers for a single stream. To prevent this being used by an attacker to attack Snort_inline, I wanted give the admin the option to set a maximal wscale size.

Snort_inline and out of order packets

Mon, 30 Jul 2007 21:22:56 +0000

In Snort_inline’s stream4 modifications, one of the changes is that out of order TCP packets are treated differently from unmodified stream4. This can cause some new alerts to appear and some unexpected behaviour. So I’ll try to explain what happens here.

First of all let me explain quickly what out of order packets are. To put it simple, TCP packets are send out by the source host in a specific order but can arrive in a different order at the destination. Packetloss, link saturation, routing issues are among many things that can cause this. A Snort_inline specific issue is that when Snort_inline can’t keep up with the packets it needs to process, it will drop packets which causes packetloss. These packets will then have to be resent by the sending host.

TCP Window scaling in Snort_inline

Fri, 15 Jun 2007 22:04:57 +0000

The TCP window field in the TCP header is only 16 bits, so the maximum window size it can handle is only 64kb. A long time ago this was enough, but nowadays it isn’t, by far. Luckily, this is something the window scaling option fixes. Window scaling is very common these days. Your pc or laptop probably uses it by default. Snort’s stream4 however, does not support it. This means that when tracking and reassembling streams, Snort for most connections has no idea about what data is in window and which is out of window. To make matters worse, the packets that are in window when using wscaling, but appear out of window when the wscaling is not accounted for, are never used in the reassembly process. This makes Snort evadable.

Memory leak fixed in stream4inline

Tue, 22 May 2007 21:54:17 +0000

A few days ago William told me that if he enabled stream4inline on a busy gateway, Snort_inline would consume all memory within hours. The problem went away when disabling stream4inline, so it made sense that the problem would be in there somewhere.

The first suspect was the reassembly cache. The reassembly cache is used to keep a per stream copy of the reassembled packet in memory. While being memory expensive, it greatly speeds up the sliding window stream reassembly process, especially with small packets. The reason for this being the first and primary suspect is that this is the only place where stream4inline code allocates memory. Reviewing the code however, showed no leaks and adding a debug counter to monitor the memory usage also showed that the leak was not in that code.

Snort_inline and TCP Segmentation Offloading

Fri, 20 Apr 2007 16:36:55 +0000

Since a short while I have a gigabit setup at home. My laptop has a e1000 Intel NIC, my desktop a Broadcom NIC.While playing with Snort_inline and netpipe-tcp, I noticed something odd. I got tcp packets that had the ‘Don’t Fragment’ option set, but were still bigger than the mtu size of the link. Snort_inline read packets of up to 26kb from the queue, and wireshark and tcpdump were seeing the packets as well. This was only for outgoing packets on the e1000 NIC. The receiving pc saw the packets split up in multiple packets that were honoring the mtu size. This got me thinking that some form of offloading must be taking place and indeed this was the case:

Snort_inline patch updated to 2.6.1.2

Wed, 17 Jan 2007 11:55:34 +0000

With the recent Snort vulnerabilities we had to make a choice if we would backport the fixes to our Snort_inline 2.6.0.2 patch or that we would upgrade to 2.6.1.2. Upgrading makes most sense since SourceFire improves Snort with every release, but since the upgrade process has been very painful the last couple of releases, we weren’t really looking forward to it.

Earlier I wrote about my testing with Subversion for Snort_inline, and I found out that using Subversion made the upgrade procedure much easier and much less time consuming. So upgrading it was. Generally there were little changes to the Snort_inline patch required.

Snort_inline 2.6 development update

Sat, 23 Dec 2006 00:35:58 +0000

Development of Snort_inline 2.6 experienced a bit of a setback when William and I discovered that the new Stream4inline had some issues with detecting certain attacks. Since we are scanning the reassembled stream certain detection plugins didn’t work as expected. Basically every detection plugin that uses absolute offsets from the packet start is messed up when we scan the reassembled stream only.

This is because the start of the reassembled stream doesn’t match with the start of the last packet added to this stream. Most TCP sigs are using offsets match against the start of the stream, or relative matches. For example a rule like:

Update on Snort_inline 2.6.0.2 development

Fri, 10 Nov 2006 11:54:11 +0000

I have spend the last week trying to find a very annoying bug that caused Snort_inline to go into 100% CPU on certain traffic. It kept working, only my P3 500Mhz home gateway slowed down to between 2kb/s and 25kb/s, while normally it handles the full 325kb/s for my DSL line at around 25% CPU.

Snort comes with a number of performance measurement options. In 2.6 –enable-perfprofiling was introduced. Also, –enable-profile builds Snort for use with gprof. Next to those you can use strace and ltrace with the -c option to see the ammount of time spend in the several functions.

I already knew the problem was related to my new Stream4 code, since running Snort_inline without the ‘stream4inline’ option made the problem go away. So my performance debugging and code reviews were focussed on that code. However, the performance statistics showed no functions that took large ammounts of time in Stream4.

Snort_inline: running Snort_inline 2.6.0.2

Thu, 05 Oct 2006 08:13:29 +0000

No, it’s not released. But it wil be soon… really!

William has done most of the hard work of porting our Snort_inline patch from 2.4.5 to 2.6. I have mostly been working on improving the stream4inline modification. I have written about this before. Like the stream4inline modification in Snort_inline 2.4.5 it scans the stream in a sliding window, making it possible to drop an attack detected in the reassembled stream. The new code does the same but is much faster, at the cost of higher memory usage.

Snort_inline: idea for an improved bait-and-switch

Mon, 10 Jul 2006 22:13:09 +0000

William Metcalf recently wrote a bait-and-switch plugin for Snort_inline. The idea is that when a rule matches on certain traffic this plugin loads an iptables rule into the system that redirects the offending host to another server. This can present the user an error message such as “Access Denied” for example, but this server can also have al kinds of sniffing tools, or even be a honeypot.

As the plugin currently creates an iptables rule it only works with linux. Also, it has some difficulty with existing iptables rulesets that might be maintained by other programs, such as my own Vuurmuur. My idea is to investigate whether or not it is possible to simply do the redirection in Snort_inline itself. By rewriting the ipaddress in the IP header, it might work as well. Naturally, this would need to be done for every packet, but with a connection to either the flow engine or the stream engine, this should be able to work… just a thought…

Snort_inline: Adapting the TCP stream reassembler

Mon, 10 Jul 2006 16:34:25 +0000

Currently I am rewriting a modification of the TCP reassembler in Snort_inline. Snort’s TCP reassembler is called Stream4 and it works fairly well in IDS mode, however it has some serious issues in inline mode. The biggest and most important issue is that Snort_inline cannot block an attack if it is detected in the reassembled stream. In Snort_inline 2.4 we made our first attempt to fix this with the stream4inline modification.