Comments on: Improving Snort_inline’s NFQ performance

By: Dave R

Dave R — Tue, 12 Feb 2008 23:24:56 +0000

You’re right - I’d forgotten that you can do it that way. Actually, I expose the queue_max in /sys, so I just change it once and forget it 8-).

By: Victor Julien

Victor Julien — Mon, 11 Feb 2008 14:11:15 +0000

Thanks for your explanations Dave. I think the number of outstanding packets nfqueue can deal with can also be changed with the –queue-maxlen option. It uses the nfq_set_queue_maxlen() call for that…

By: Dave R

Dave R — Sun, 10 Feb 2008 23:14:09 +0000

D’oh! Forgot to mention the other side of the max packets on a queue versus memory conundrum…. The number of packets allowed to be outstanding on a netfilter_queue. Which is 1024. So…. Which ever you run into first (max packets outstanding) or out of memory to put ‘em in is what bites…

To change the number of outstanding packets on an NFQUEUE isn’t as easy as a sysctl -w, though - you need to modify nfnetfilter_queue.c and recompile it to change the value:

#define NFQNL_QMAX_DEFAULT 1024

Here’s one way to look at it - each outstanding packet takes up one of the above QMAX spots, and some amount of memory. If your acerage traffic is 600 bytes in size, you can use up something like 650K (I think there’s a little overhead per packet) of memory. For full size TCP streams and acks the average packet size is more like 1100, so you’re looking at more like 1.2MB of buffer. If you’re using jumbo packets, order more RAM now 8-).

What you’re trying to buy by adjusting the QMAX and rmem_default settings upward is breathing space for snort, when that new PCRE rule you wrote this afternoon takes waaaaaaayyyyyy longer to match than you thought it would, or you just get slammed with a DOS attack….

By: Dave R

Dave R — Tue, 05 Feb 2008 16:17:17 +0000

The line
sysctl -w net.core.rmem_default=’8388608′
tells the kernel to allow up to 8388608 bytes worth of (received) packets to be queued for every open socket in the system. It’s dynamically allocated from free memory (Low Memory on 32 bit kernels). You *don’t* want to run out of Low Memory on a 32 bit kernel… (cat /proc/meminfo). Bad things happen - the OOM killer goes on a rampage, and random process get killed (unless protected against OOM killing (echo “-17″ > /proc/pid/oom_adj)).

So the buffer sizes allocated have to be weighed against the possibility that under duress, the kernel will have too much memory used up in outstanding receive buffers, and start freeing things up in an unpleasant manner. Tuning the kernel for these condtions is an art, not a science…