postfix-users October 2010 archive
Main Archive Page > Month Archives  > postfix-users archives
postfix-users: Re: postfix doubling emails and spam!

Re: postfix doubling emails and spam!

From: Al Zick <al_at_nospam>
Date: Fri Oct 29 2010 - 19:39:27 GMT
To: postfix users <postfix-users@postfix.org>

Hi,

On Oct 27, 2010, at 11:50 PM, Noel Jones wrote:

> On 10/27/2010 7:02 PM, Al Zick wrote:
>> Is there a replacement for procmail? I know it seemed to take
>> longer and did raise cpu usage, but when I first installed it
>> with bogofilter, it almost eliminated spam getting into my inbox.
>
> depends on why you're using procmail... If you need a way to
> interface spam/virus filtering, amavisd-new + spamassassin + clamav
> + sanesecurity clam signatures are a popular and effective
> combination, although SpamAssassin can use quite a bit of resources.

Currently, I just use procmail to interface with the spam filters. I
would really like to put a bunch of rules into procmail too, for
example: if is sees the word viagra anywhere in the email, it is
spam, there is no reason to go any further with it.

Right now, I am concerned that I would need a quad core, quad
processor system that was dedicated to just running spamassassin, so
I am looking at other solutions.

>>>> problems lately have been with email. I feel like I need to
>>>> get postfix to stop using so much cpu.
>>>
>>> Show some evidence. Postfix shouldn't use very much CPU.
>>
>>
>> per second hitting the mail server just to be temporarily
>> bounced by the graylisting when in the end they get bounced
>> anyway. Even after they are bounced, they just keep coming
>> anyway.
>>
>
> Most greylist services use DEFER_IF_PERMIT so that mail that can be
> permanently rejected is not deferred to retry.

I think that I need to accept and delete email that is being sent to
maybe the top few email address that don't exist and never had
existed. They add the most lines to the log. When I was just
accepting them and deleting them, then the log was very quiet.

> If your forwarded mail is what's attempting repeated delivery
> despite being rejected, you'll need to whitelist those servers and
> eat the mail. Otherwise, firewall clients who refuse to go away.

I will definitely be whitelisting all the servers that forward email
to me. I will also be whitelisting all my friend's mail servers. This
will probably help with a lot of the bounce rebouncing.

> Identify the problem, then address it

>>> Sounds as if you've foolishly set "soft_bounce = yes"
>>
>> # postconf -d | grep soft_bounce
>> soft_bounce = no
>
>
> "man postconf" to see what "-d" does and why the above information
> is useless.
>
> But no matter; soft_bounce doesn't appear in your "postconf -n"
> listing, so that's not it.

Is there anything else that could cause a soft_bounce?

> [postconf output]
>> bounce_queue_lifetime = 2d
>> default_destination_concurrency_limit = 5
>> default_process_limit = 15
>> maximal_backoff_time = 4h
>> maximal_queue_lifetime = 3d
>> minimal_backoff_time = 2h
>> qmgr_message_active_limit = 50
>> qmgr_message_recipient_limit = 50
>> queue_run_delay = 30m
>
> Your settings resemble what someone with an underpowered server
> with a bad backscatter problem might try. If that's not your
> situation, use the defaults. If that *is* your situation, address
> the source of the problem rather than putting postfix colored band-
> aids on it.

What exactly is a backscatter problem?

If I do have a backscatter problem, what should the settings be?

> Mucking around with the above settings is a good way to cripple
> postfix performance. Tread carefully here.
>
> With a process limit of 15, any server less than 10 years old
> should hardly get above idle. The default has been 100 for years;
> most servers can easily support several times that.

This install of postfix is from a few years ago and it was not up to
date then (it is what installed with the OS and I never updated it).
A friend of mine recompiled OS for better optimization. I think it
was already pretty old when I install it. Really, I was supposed to
upgrade Postfix through the packaging system because there was some
known problems with what came with the OS, but I never did. I had a
friend of mine look at it because it would not receive or send emails
to the outside world, and I am not really sure what he did anymore. I
think he added one line to master.cf and I think he had me make other
changes to master.cf (although, he may have made them). I do remember
that the server would basically not work at all and I think the
process limit was set to something lower and I raised it to 15. This
server runs a lot of other things, like 2 web servers, named, squid,
and a whole lot of custom written software, and it pretty much does
everything that both of my other dedicated servers do, so that may be
why it was set so low.

Could this be one of the reasons I see so many bounces in the log?
Would this act like a soft bounce? Besides the process limit what
else should be raised?

>> smtpd_recipient_restrictions = permit_mynetworks,
>> reject_unauth_destination, reject_invalid_hostname,
>> reject_unauth_pipelining, reject_non_fqdn_sender,
>> reject_unknown_sender_domain, reject_non_fqdn_recipient,
>> reject_unknown_recipient_domain, reject_rbl_client
>> bl.spamcop.net, reject_rbl_client cbl.abuseat.org, permit
>
> OK. I suggest dropping cbl.abuseat.org and adding zen.spamhaus.org
> (zen includes cbl data).

I was using zen.spamhaus.org, but it seemed to create too many false
pastives. Many emails I was not getting and it was making people mad.
This is when things really started to become a problem, I started
getting duplicate emails, although I can't find anything in the
Postfix log. I just started procmail logging, so I will see if it
shows anything. I think spammers are sending emails that cause this
problem, but I am not sure.

> I also like using reject_unknown_reverse_client_hostname to reject
> zombies with no rDNS record.
> http://www.postfix.org/postconf.
> 5.html#reject_unknown_reverse_client_hostname

I will look at this. I know from looking at the headers that some
servers that should be able to send email to me will not be able to
if I use this. Is there a way that I can whitelist servers from this?

>> unknown_local_recipient_reject_code = 550
>
> Good.
>
> Consider a lower smtpd_hard_error_limit so that postfix can
> disconnect misbehaving clients sooner. Something between 2..10 is
> probably good for most sites.
> http://www.postfix.org/postconf.5.html#smtpd_hard_error_limit

This is something that I will definitely put into my config.

> Consider using the postfix anvil service to limit how much mail
> individual clients can send. Note: anvil is not for traffic
> shaping. You may need to exempt a few high volume clients, such as
> your forwarders.
> http://www.postfix.org/TUNING_README.html#conn_limit

> If you have repeat offenders that send lots of spam, firewall
> them. You can use fail2ban to automatically temporarily blacklist
> clients that exceed a set number of rejects per time period.
> http://www.fail2ban.org

Right now, I don't have too many people who relay email through my
server, although it does relay the email for my other servers. I just
don't give anyone new a chance to send spam, because at one time I
had a problem with this.

> If system load is a problem, consider running a recent postfix
> snapshot with the new postscreen service. The intent of postscreen
> is to reject as much spam as possible using as few system resources
> as possible. (You'll need to upgrade somewhat frequently to stay
> with reasonably current snapshots.)
> http://www.postfix.org/POSTSCREEN_README.html

I will look into this. This may be a great solution. The link doesn't
seem to work right now.

>
>
>> I was wondering if using something like policyd would help the
>> spam problem?
>
> Your time will probably be best spent in identifying the actual
> problem and addressing it, rather than just bolting a bunch of
> stuff into postfix hoping something will change.
>
> Once you identify the problem as something policyd might help, then
> policyd is worth trying.

Right now, the big issue is spam, somewhere some emails are being
duplicated, and the fact that my postfix log looks like a war zone.
On average I don't get a lot of connections, but at times I get what
I would consider a real high number in a very short time (I think
these are an attempt to overload the server). It is not so much that
it is overloaded as I am tired of all the bogus connections and I
really think I need to deal with them better.

>> Is there a proper way to filter spam? If so, what is it?
>
> If it was easy, no one would get spam. This situation is
> complicated since the type of spam and the tolerance for false
> positives are local issues. Sounds as if a lot of your spam is
> forwarded from accounts on other servers; that's something
> SpamAssassin and clamav+sanesecurity sigs can help with.
>
> You can have great success if you can spend time and energy on it;
> otherwise just sign up for google apps and gmail.

I spend a lot of time trying to deal with spam. What I have found is
that I need to update my spam filtering often, but still I seem to
need to totally revamp the way that I am dealing with spam. I can't
seem to get away with a lot of false positives, yet I don't want to
deliver the amount of spam that I have been.

I have several websites that I own that are in the top 1,000,000
sites based on traffic according to Alexa and although this server
only hosts the email for like 30 some domains. I seem to get more
than my fair share of spam. Right now, it is still manageable, but
soon I will need a very high end dedicated mail server, if I don't
change something. Personally, I feel my config is wrong and that is
why I am asking some questions.

I was also looking at something else and it looks like Postfix was
built without pcre. Will I be able to use header checks without this?

Sincerely,
Al