Re: [Snort-users] Unusual Snort performance stats

From: Matt Watchinski <mwatchinski_at_nospam>
Date: Mon Feb 22 2010 - 16:29:14 GMT
To: Willst Mail <willstmail@gmail.com>

Couple things to look at that might help track this down.

Outstanding means that packets never got out of the ethernet card before they got dropped. IE pcap didn't get to them before they disappeared. One thing to check here is the type of ethernet card you are using and if you are running the latest drivers for that card. I normally see best performance out of Intel cards. Obviously the stats are borked for some reason since they are > 100%, but its something to consider.
Feb 22 03:30:12 snortsensor1 snort[4567]: OTHER: 72674000 (18.281%)

This stats means that some percentage of your traffic contains protocols that snort doesn't do anything with. Tracking these down and add BPF's to ignore them could improve performance.

3. Are you using CPU affinity to lock the snort process to a specific CPU? If not this is something to try. If snort bounces to another CPU then the cache line is reset and performance can suffer.

4. Upgrade to snort 2.8.5.3, just in case.

Cheers,
-matt

On Mon, Feb 22, 2010 at 8:37 AM, Willst Mail <willstmail@gmail.com> wrote:

> Hello,
> We are seeing some strange statistics in Snort stats. When I look at
> syslog following a restart of Snort, this is what I see for
> performance:
>
> Feb 22 03:30:12 snortsensor1 snort[4567]: Run time prior to being
> shutdown was 86389.32948 seconds
> Feb 22 03:30:12 snortsensor1 snort[4567]:
>
> ===============================================================================
> Feb 22 03:30:12 snortsensor1 snort[4567]: Packet Wire Totals:
> Feb 22 03:30:12 snortsensor1 snort[4567]: Received: 201246607
> Feb 22 03:30:12 snortsensor1 snort[4567]: Analyzed: 396896876
> (197.219%)
> Feb 22 03:30:12 snortsensor1 snort[4567]: Dropped: 2798169
> (1.390%)
> Feb 22 03:30:12 snortsensor1 snort[4567]: Outstanding:
> 18446744073511103178 (9166238551048.516%)
> Feb 22 03:30:12 snortsensor1 snort[4567]:
>
> ===============================================================================
> Feb 22 03:30:12 snortsensor1 snort[4567]: Breakdown by protocol
> (includes rebuilt packets):
> Feb 22 03:30:12 snortsensor1 snort[4567]: ETH: 397535427 (100.000%)
> Feb 22 03:30:12 snortsensor1 snort[4567]: ETHdisc: 0 (0.000%)
> Feb 22 03:30:12 snortsensor1 snort[4567]: VLAN: 238577055 (60.014%)
> Feb 22 03:30:12 snortsensor1 snort[4567]: IPV6: 0 (0.000%)
> Feb 22 03:30:12 snortsensor1 snort[4567]: IP6 EXT: 0 (0.000%)
> Feb 22 03:30:12 snortsensor1 snort[4567]: IP6opts: 0 (0.000%)
> Feb 22 03:30:12 snortsensor1 snort[4567]: IP6disc: 0 (0.000%)
> Feb 22 03:30:12 snortsensor1 snort[4567]: IP4: 397110659 (99.893%)
> Feb 22 03:30:12 snortsensor1 snort[4567]: IP4disc: 18109940 (4.556%)
> Feb 22 03:30:12 snortsensor1 snort[4567]: TCP 6: 0 (0.000%)
> Feb 22 03:30:12 snortsensor1 snort[4567]: UDP 6: 0 (0.000%)
> Feb 22 03:30:12 snortsensor1 snort[4567]: ICMP6: 0 (0.000%)
> Feb 22 03:30:12 snortsensor1 snort[4567]: ICMP-IP: 0 (0.000%)
> Feb 22 03:30:12 snortsensor1 snort[4567]: TCP: 270400570 (68.019%)
> Feb 22 03:30:12 snortsensor1 snort[4567]: UDP: 34726192 (8.735%)
> Feb 22 03:30:12 snortsensor1 snort[4567]: ICMP: 1380875 (0.347%)
> Feb 22 03:30:12 snortsensor1 snort[4567]: TCPdisc: 3 (0.000%)
> Feb 22 03:30:12 snortsensor1 snort[4567]: UDPdisc: 292 (0.000%)
> Feb 22 03:30:12 snortsensor1 snort[4567]: ICMPdis: 0 (0.000%)
> Feb 22 03:30:12 snortsensor1 snort[4567]: FRAG: 126408 (0.032%)
> Feb 22 03:30:12 snortsensor1 snort[4567]: FRAG 6: 0 (0.000%)
> Feb 22 03:30:12 snortsensor1 snort[4567]: ARP: 95294 (0.024%)
> Feb 22 03:30:12 snortsensor1 snort[4567]: EAPOL: 0 (0.000%)
> Feb 22 03:30:12 snortsensor1 snort[4567]: ETHLOOP: 25887 (0.007%)
> Feb 22 03:30:12 snortsensor1 snort[4567]: IPX: 0 (0.000%)
> Feb 22 03:30:12 snortsensor1 snort[4567]: OTHER: 72674000 (18.281%)
> Feb 22 03:30:13 snortsensor1 snort[4567]: DISCARD: 18110235 (4.556%)
> Feb 22 03:30:13 snortsensor1 snort[4567]: InvChkSum: 2915 (0.001%)
> Feb 22 03:30:13 snortsensor1 snort[4567]: S5 G 1: 219175 (0.055%)
> Feb 22 03:30:13 snortsensor1 snort[4567]: S5 G 2: 365456 (0.092%)
> Feb 22 03:30:13 snortsensor1 snort[4567]: Total: 397535427
> Feb 22 03:30:13 snortsensor1 snort[4567]:
>
> ===============================================================================
> Feb 22 03:30:13 snortsensor1 snort[4567]: Action Stats:
> Feb 22 03:30:13 snortsensor1 snort[4567]: ALERTS: 911
> Feb 22 03:30:13 snortsensor1 snort[4567]: LOGGED: 911
> Feb 22 03:30:13 snortsensor1 snort[4567]: PASSED: 0
> Feb 22 03:30:13 snortsensor1 snort[4567]:
>
> ===============================================================================
>
> Somehow we are analyzing 197% of packets, and we have a remarkable
> number of outstanding packets (I'm not even sure what "outstanding"
> packets are). We do a restart of Snort every 24 hours, and these
> stats are pretty typical. We are consistently around 193-198%
> analyzed, and some ridiculous number for outstanding (1844.... never
> the same). This is v2.8.5.1 (build 114), configured with
> "--enable-sourcefire --enable-targetbased --enable-perfprofiling."
> Snort is running as a daemon and is listening on a single interface.
> That interface is receiving the transmit and receive lines from
> multiple ISP links (eventually we will be adding additional sensors so
> each only monitors a single ISP link). We haven't done much rule
> tuning yet, and the only output directive is for unified2. This
> machine has 4gb RAM and 8 cores (dual quad-core 3.0gHz Xeons?) with
> Snort typically using around 40% of one core.
>
> Any ideas why our performance statistics could be so unusual?
>
>
> ------------------------------------------------------------------------------
> Download Intel® Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> Snort-users mailing list
> Snort-users@lists.sourceforge.net
> Go to this URL to change user options or unsubscribe:
> https://lists.sourceforge.net/lists/listinfo/snort-users
> Snort-users<https://lists.sourceforge.net/lists/listinfo/snort-users%0ASnort-users>list archive:
> http://www.geocrawler.com/redir-sf.php3?list=snort-users
>
-- Matthew Watchinski Sr. Director Vulnerability Research Team (VRT) Sourcefire, Inc. Office: 410-423-1928 http://vrt-sourcefire.blogspot.com && http://www.snort.org/vrt/

------------------------------------------------------------------------------ Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev

_______________________________________________ Snort-users mailing list Snort-users@lists.sourceforge.net Go to this URL to change user options or unsubscribe: https://lists.sourceforge.net/lists/listinfo/snort-users Snort-users list archive: http://www.geocrawler.com/redir-sf.php3?list=snort-users

This message: [ Message body ]
Next message: Sharma, Ashish: "Re: [Snort-users] Unable to run Snort in IPS mode"
Previous message: Ryan Jordan: "Re: [Snort-users] Unusual Snort performance stats"
In reply to: Willst Mail: "[Snort-users] Unusual Snort performance stats"
Next in thread: Jason Haar: "Re: [Snort-users] Unusual Snort performance stats"
Reply: Jason Haar: "Re: [Snort-users] Unusual Snort performance stats"

Contemporary messages sorted: [ By Date ] [ By Thread ] [ By Subject ] [ By Author ]