postfix-users October 2010 archive
Main Archive Page > Month Archives  > postfix-users archives
postfix-users: Re: postfix doubling emails and spam!

Re: postfix doubling emails and spam!

From: Al Zick <al_at_nospam>
Date: Thu Oct 28 2010 - 00:02:32 GMT
To: postfix users <postfix-users@postfix.org>

Hi Noel,

On Oct 27, 2010, at 5:15 PM, Noel Jones wrote:

> On 10/27/2010 4:44 PM, Al Zick wrote:
>> Hi,
>>
>> On Oct 27, 2010, at 12:38 PM, Noel Jones wrote:
>>
>>> On 10/27/2010 10:37 AM, Al Zick wrote:
>>>> Hi,
>>>>
>
> Turn off verbose logging. That may help your CPU usage too.
ok.

>>> Not likely. A broken alias is the first guess. What did you
>>> change?
>>
>> I didn't change anything and I can't find any duplicates in
>> the log. I have to wonder if the problem didn't occur after it
>> was delivered to procmail.
>
> A procmail delivery problem wouldn't surprise me.

Is there a replacement for procmail? I know it seemed to take longer
and did raise cpu usage, but when I first installed it with
bogofilter, it almost eliminated spam getting into my inbox.

I just turned on procmail logging, so maybe I will get some answers
there.

>>>> I then have postfix pass the email to procmail where it is
>>>> filtered with bogofilter. I keep giving bogofilter more spam
>>>> to look at, but it doesn't seem to block all the spam anymore,
>>>> although it blocks some spam. When I first installed it,
>>>> bogofilter worked very well.
>>>
>>> Sounds as if bogofilter is poorly trained. Ask for help on a
>>> bogofilter forum, or just delete the database and start over.
>>
>>
>> I have deleted the database many times and started over. If I
>> delete the older spam and the spam that is out of order being
>> sorted by date it will work again for a while.
>
> If bogofilter isn't working for you, try another tool. But
> remember that nothing will work long term without some attention.

I really don't know how much more attention I can give bogofilter. I
give it new spam to look at every few days, although recently I have
given up on it.

>> problems lately have been with email. I feel like I need to
>> get postfix to stop using so much cpu.
>
> Show some evidence. Postfix shouldn't use very much CPU.

Over the last month or so I am getting something like 2 to 3 times
the spam, so Postfix cpu usage has changed with it. It is not that it
is using an abnormal amount of cpu for the increased work it is
doing. I am sorry I should have explained that before.

>> each mail server sending 100's of emails. When I used to use
>> postgray (I gave that up years ago), postgray would use all
>> the cpu.
>
> So you have a 200MHz 386 or something?

So then you are saying it is normal to have 1,000s of emails per
second hitting the mail server just to be temporarily bounced by the
graylisting when in the end they get bounced anyway. Even after they
are bounced, they just keep coming anyway.

>> The problem is these same emails continue to be bounced by my
>> mail server. If I just let them be delivered then it does
>
> Sounds as if you've foolishly set "soft_bounce = yes"

# postconf -d | grep soft_bounce
soft_bounce = no

> This would be a good time to share your "postconf -n" output and
> the contents of master.cf

alias_database = hash:/etc/postfix/aliases
alias_maps = hash:/etc/postfix/aliases
bounce_queue_lifetime = 2d
canonical_maps = hash:/etc/postfix/canonical.sender
command_directory = /usr/sbin
config_directory = /etc/postfix
daemon_directory = /usr/libexec/postfix
debug_peer_level = 2
default_destination_concurrency_limit = 5
default_process_limit = 15
disable_vrfy_command = yes
html_directory = /usr/share/doc/html/postfix
inet_protocols = all
mail_owner = postfix
mailbox_command = /usr/local/bin/procmail
mailbox_size_limit = 512000000
mailq_path = /usr/bin/mailq
manpage_directory = /usr/share/man
maximal_backoff_time = 4h
maximal_queue_lifetime = 3d
minimal_backoff_time = 2h
mydestination = $myhostname, $mydomain, localhost.$mydomain
mydomain = datazap.net
myhostname = agnus.datazap.net
mynetworks = 127.0.0.0/8
newaliases_path = /usr/bin/newaliases
qmgr_message_active_limit = 50
qmgr_message_recipient_limit = 50
queue_directory = /var/spool/postfix
queue_run_delay = 30m
readme_directory = /usr/share/examples/postfix
sample_directory = /usr/share/examples/postfix
sender_canonical_maps = hash:/etc/postfix/canonical.sender
sendmail_path = /usr/sbin/sendmail
setgid_group = maildrop
smtpd_helo_required = yes
smtpd_recipient_restrictions = permit_mynetworks,
reject_unauth_destination, reject_invalid_hostname,
reject_unauth_pipelining, reject_non_fqdn_sender,
reject_unknown_sender_domain, reject_non_fqdn_recipient,
reject_unknown_recipient_domain, reject_rbl_client
bl.spamcop.net, reject_rbl_client cbl.abuseat.org, permit
strict_rfc821_envelopes = yes
unknown_local_recipient_reject_code = 550
virtual_alias_maps = hash:/etc/postfix/virtual

smtp inet n - n - - smtpd
#submission inet n - n - - smtpd
# -o smtpd_enforce_tls=yes
# -o smtpd_sasl_auth_enable=yes
# -o smtpd_client_restrictions=permit_sasl_authenticated,reject
#smtps inet n - n - - smtpd
# -o smtpd_tls_wrappermode=yes
# -o smtpd_sasl_auth_enable=yes
# -o smtpd_client_restrictions=permit_sasl_authenticated,reject
#628 inet n - n - - qmqpd
pickup fifo n - n 60 1 pickup
cleanup unix n - n - 0 cleanup
qmgr unix n - n 300 1 qmgr
#qmgr fifo n - n 300 1 oqmgr
tlsmgr unix - - n 1000? 1 tlsmgr
rewrite unix - - n - - trivial-rewrite
bounce unix - - n - 0 bounce
defer unix - - n - 0 bounce
trace unix - - n - 0 bounce
verify unix - - n - 1 verify
flush unix n - n 1000? 0 flush
proxymap unix - - n - - proxymap
smtp unix - - n - - smtp
# When relaying mail as backup MX, disable fallback_relay to avoid MX
loops
relay unix - - n - - smtp
         -o fallback_relay=
# -o smtp_helo_timeout=5 -o smtp_connect_timeout=5
showq unix n - n - - showq
error unix - - n - - error
retry unix - - n - - error
discard unix - - n - - discard
local unix - n n - - local
virtual unix - n n - - virtual
lmtp unix - - n - - lmtp
anvil unix - - n - 1 anvil
scache unix - - n - 1 scache

Do you see anything that is mis-configured or that I could change to
help performance?

>> lower the amount of mail attempted to be sent to it by like
>> 90%. I would have never though of this idea, but I read an
>> ariticle online on how to stop fighting the battle on spam and
>> winning the war. This is one of the things they recommend.
>
> This is bad advice. Not everything you read on the internet is
> helpful.

True enough.

>>> Very bad idea. Reject mail you don't intend to deliver.
>>
>> I thought that someone would say that and really I am not sure
>> how to deal with the over flow of spam. Although, I can see
>> how just letting it go through and not fighting it only to
>> have a filter look at it later may work.
>
> Nothing is more efficient than rejecting the mail before it's
> accepted. Period.

I was wondering if using something like policyd would help the spam
problem?

>> I would really like to filter email after the mail server
>> passes it to procmail, because I have noticed email that is
>> forwarded from another mail server the filtering in Postfix
>> doesn't seem to do anything for it, it may just as well
>> immediately give it to procmail for filtering. Also, I have
>> considered using something like fetchmail to get the mail from
>> other mail servers and then passing it through procmail for
>> filtering. The forwarded emails seem to be one of the reasons
>
>
> So get rid of the forwarded accounts.

I won't get that email then. It would just be nice if there was a
better way to filter it. I have thought about writing our own spam
filtering software to run on the server, although I could do most of
the same things with a set of rules for procmail.

Is there a proper way to filter spam? If so, what is it?

Thanks,
Al