spamassassin-users June 2010 archive
Main Archive Page > Month Archives  > spamassassin-users archives
spamassassin-users: How do I get better processing/delivery time

How do I get better processing/delivery times?

From: Spiro Harvey <spiro_at_nospam>
Date: Wed Jun 09 2010 - 23:51:21 GMT
To: spamassassin-user list <users@spamassassin.apache.org>

I maintain a mail cluster that gets about 70,000 messages a day per
node.

I'm just wondering if it's possible to decrease the scan times. In the
TOTALS section AvgTm is the average "scantime" in the spamassassin log
file:

(Delivered are messages that SA scores under 5, Spamboxed are scored
5+, but under 10, and Rejected are 10+)

# ./knl-spam-stats.awk /var/log/spamassassin.*

 TOTALS
                              AvgTm AvgThruput
             # Msgs %/Total (sec) (bytes/sec)
             ~~~~~~ ~~~~~~~~~ ~~~~~ ~~~~~~~~~~~
  Delivered 176086 ( 17.80%) 15.22 2208.40
  Spamboxed 51194 ( 5.17%) 19.92 550.14
  Rejected 762189 ( 77.03%) 19.30 537.56

  Total 989469 messages processed

  (70676/day; 2944.85/hr; 49.08/min; 0.82/sec)

 BLACKLIST HITS
  Blacklist Msgs %/Total %/Spam Avg Score
  ~~~~~~~~~~~~~ ~~~~~~ ~~~~~~~~~ ~~~~~~~~ ~~~~~~~~~
  Spamhaus SBL 2087 ( 0.21%) ( 0.26%) 16.15
  Spamhaus PBL 569825 ( 57.59%) ( 70.06%) 21.99
  Spamhaus XBL 497403 ( 50.27%) ( 61.15%) 21.98
  SBL URI 187292 ( 18.93%) ( 23.03%) 26.05
  NJABL 3544 ( 0.36%) ( 0.44%) 23.65
  SORBS 387539 ( 39.17%) ( 47.65%) 22.32
  Spamcop 513748 ( 51.92%) ( 63.16%) 22.48
  SURBL URI 360620 ( 36.45%) ( 44.34%) 27.25
  RFC Ignorant 29295 ( 2.96%) ( 3.60%) 20.68

 CUSTOM RULE HITS
  Custom Rule Msgs %/Total Avg Score
  ~~~~~~~~~~~~~ ~~~~~~ ~~~~~~~~~ ~~~~~~~~~
  MIME/JPG 84 ( 0.01%) 15.45
  ZIP file 741 ( 0.07%) 18.44

Yet, on another mail cluster that only gets 4-5000 messages a day per
node, the average scantimes are 4-5 seconds. Both have the same custom
rules, so any slowness in processing regexes should be noticable on
both systems.

In the first case, we have started rsyncing Spamhaus' blacklists in the
hopes that it would increase scantimes by decreasing DNS lookup times.
It hasn't really made too much difference, but my main concern is that
the messages seem to be taking so long regardless.

The boxes are running Sendmail 8.14 + ClamAV 0.96 + SA 3.3.1 + Razor

SPAMDOPTIONS="-d -x -c -m50 -H -s local2 /home/spamd -u spamd
--min-children=10 --min-spare=10"

Core 2 Duo @2.93GHz, 4GB RAM. Load averages typically sit at 5-7 during
the day.

Any advice on how I can tune the scantimes?

-- Spiro Harvey Knossos Networks Ltd 021-295-1923 www.knossos.net.nz