You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Luis Croker <lc...@megacable.com.mx> on 2008/11/05 19:58:25 UTC
Re: Spamassassin+amavis
Hi all...
I was doing some tests with all the recommendations you sent me...
and I can make to work the server correctly... I was filtering spam with
no problems and my performances troubles dissapeard...
I just configured 5 procs for amavis and postfix content filter and
I turn off the network tests... the server can filter a lot of spam and
delivery quickly... but now appears another problem :(
Until today morning... I was filtering and deliverying fine, but
suddenly I received these messages and the delivery is sooo slow and
the mail queue just is growing and growing....
Nov 5 12:51:23 mailgw postfix/qmgr[14251]: warning: mail for
[127.0.0.1]:10024 is using up 4001 of 4004 active queue entries
Nov 5 12:51:23 mailgw postfix/qmgr[14251]: warning: you may need to
reduce smtp-amavis connect and helo timeouts
Nov 5 12:51:23 mailgw postfix/qmgr[14251]: warning: so that Postfix
quickly skips unavailable hosts
Nov 5 12:51:23 mailgw postfix/qmgr[14251]: warning: you may need to
increase the main.cf minimal_backoff_time and maximal_backoff_time
Nov 5 12:51:23 mailgw postfix/qmgr[14251]: warning: so that Postfix
wastes less time on undeliverable mail
Nov 5 12:51:23 mailgw postfix/qmgr[14251]: warning: you may need to
increase the master.cf smtp-amavis process limit
Nov 5 12:51:23 mailgw postfix/qmgr[14251]: warning: please avoid
flushing the whole queue when you have
Nov 5 12:51:23 mailgw postfix/qmgr[14251]: warning: lots of deferred
mail, that is bad for performance
Nov 5 12:51:23 mailgw postfix/qmgr[14251]: warning: to turn off these
warnings specify: qmgr_clog_warn_time = 0
I have researched and I have founded to increase the
minimal_backoff_time and another parameters but nothing works...
Anybody have resolved someday this problem ?
thanks... Regards. :(
On Wed, 2008-10-29 at 14:59 +0000, Ned Slider wrote:
> Luis Croker wrote:
> > How can I tunr off the Network tests (RBLs) ??? Just to probe if it
> > can make the delivery faster.
> >
>
> In /etc/amavisd.conf, find the following line:
>
> $sa_local_tests_only = 0; # only tests which do not require internet
> access?
>
> and change the setting to = 1
>
> then restart amavisd
>
>
Re: Spamassassin+amavis
Posted by John Hardin <jh...@impsec.org>.
On Wed, 12 Nov 2008, Luis Croker wrote:
> turned off DCC, the network tests and install the DNS service
> locallly.
Turning off the network tests will obscure any benefit from installing a
local caching DNS server. Try turning the network tests on for a while and
see whether your performance is still poor even with the local caching DNS
server.
(...don't forget to update your /etc/resolv.conf to point at the local DNS
server so that you actually *use* it...)
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
Gun Control laws cannot reduce violent crime, because gun control
laws focus obsessively on a tool a criminal might use to commit a
crime rather than the criminal himself and his act of violence.
-----------------------------------------------------------------------
Re: Spamassassin+amavis
Posted by Luis Croker <lc...@megacable.com.mx>.
Hi this mail ius just to say thanks all the people kindly sent me a
mail trying to figure out the low performance in my server.
Right now the server is working well and filtering like I wish. The
changes I did were decrease the number of amavisd processes to 5,
turned off DCC, the network tests and install the DNS service
locallly.
Thanks all. Regards.
On Thu, 2008-11-06 at 01:07 +0100, Mark Martinec wrote:
> Luis,
>
> > I was doing some tests with all the recommendations you sent me...
> > and I can make to work the server correctly... I was filtering spam with
> > no problems and my performances troubles dissapeard...
> >
> > I just configured 5 procs for amavis and postfix content filter and
> > I turn off the network tests... the server can filter a lot of spam and
> > delivery quickly... but now appears another problem :(
>
> With your 4 CPU 4 GB mem box you should be able to run more than 4
> amavisd(+SA) processes. As a rule of a thumb, I'd say your box should
> not have trouble running 20..30 processes.
>
> > Until today morning... I was filtering and deliverying fine, but
> > suddenly I received these messages and the delivery is sooo slow and
> > the mail queue just is growing and growing....
> >
> > Nov 5 12:51:23 mailgw postfix/qmgr[14251]: warning: mail for
> > [127.0.0.1]:10024 is using up 4001 of 4004 active queue entries
>
> This is just a consequence of your amavisd+SpamAssassin not being able
> to keep up with the incoming mail flow. No fine tuning on the Postfix
> side will be able to compensate for the fact that your mail inflow rate
> is larger than the mail processing throughput of SpamAssassin filtering.
>
> What is your message rate on a normal day? Is the current mail flow
> significantly larger? Perhaps you are under a bounce storm, which can
> easily increase the mail flow rate by an order of magnitude. Examine
> what kind of messages are most typical in your mail queue (mailq, postcat),
> try to determine if these are just normal spam flow, or bounces, or
> something else (e.g. mailer abused as an open relay, perhaps by one of
> your client PCs which might have been zombiized).
>
> What is the message throughput though the filter - see what amavisd-agent
> has to report, the more interesting figures are for example:
>
> CacheAttempts 15216 3217/h 100.0 % (CacheAttempts)
> CacheHits 1750 370/h 11.5 % (CacheAttempts)
> ...
> InMsgs 15216 3217/h 100.0 % (InMsgs)
> InMsgsBounce 4176 883/h 27.4 % (InMsgs)
> InMsgsBounceKilled 3904 825/h 93.5 % (InMsgsBounce)
> ...
> TimeElapsedDecoding ...
> TimeElapsedPenPals
> TimeElapsedReceiving
> TimeElapsedSending
> TimeElapsedSpamCheck
> TimeElapsedVirusCheck
> TimeElapsedTotal
>
> How does the display of amavisd-nanny look like? Are all processes
> about evenly busy? Are processing times significantly longer than a
> couple of seconds? Set $nanny_details_level=2; (in amavisd.conf) for
> more detailed timing breakdown by amavisd-nanny.
>
> Check timing log (at log level 2), you may want to (re)confirm that
> SpamAssassin is really taking most of the time, just in case.
>
> > -I turned off DCC, Razor and Pyzor.
> > -I set the bayes use to 0.
>
> These were pretty drastic measures, significantly affecting quality
> of SA results. Once you get over the current crisis, at least put back
> the DCC and Bayes on MySQL, which are relatively low resource consumers
> compared to regexp-based rules and to Pyzor (razor is somewhere inbetween).
>
> Mark
>
>
Luis Croker
SCSA - SCNA
Administrador de Sistemas
Megacable Comunicaciones
GPG Key1024D/48C1764B
Key fingerprint = E8B6 E84F ECE4 661E 30C7 7208 042D BD09 48C1 764B
Re: Spamassassin+amavis
Posted by Duane Hill <d....@yournetplus.com>.
On Thu, 6 Nov 2008, Mark Martinec wrote:
> Luis,
>
>> I was doing some tests with all the recommendations you sent me...
>> and I can make to work the server correctly... I was filtering spam with
>> no problems and my performances troubles dissapeard...
>>
>> I just configured 5 procs for amavis and postfix content filter and
>> I turn off the network tests... the server can filter a lot of spam and
>> delivery quickly... but now appears another problem :(
>
> With your 4 CPU 4 GB mem box you should be able to run more than 4
> amavisd(+SA) processes. As a rule of a thumb, I'd say your box should
> not have trouble running 20..30 processes.
I haven't ran amavisd-new on our servers yet. However, I do run SA
directly out of Postfix on a server with 4x3.5Ghz dual core zeon
processors with 12Gb of RAM. I haven't had any issues with the following
startup params:
--min-children=10 --max-children=40 --min-spare=10 --max-spare=20
and haven't ran into any issues for over a year now. This server handles
an average over six million connections every 24 hours at present. I am
getting ready to toss more RAM at the server as connections have
increased.
Re: Spamassassin+amavis
Posted by Mark Martinec <Ma...@ijs.si>.
Luis,
> I was doing some tests with all the recommendations you sent me...
> and I can make to work the server correctly... I was filtering spam with
> no problems and my performances troubles dissapeard...
>
> I just configured 5 procs for amavis and postfix content filter and
> I turn off the network tests... the server can filter a lot of spam and
> delivery quickly... but now appears another problem :(
With your 4 CPU 4 GB mem box you should be able to run more than 4
amavisd(+SA) processes. As a rule of a thumb, I'd say your box should
not have trouble running 20..30 processes.
> Until today morning... I was filtering and deliverying fine, but
> suddenly I received these messages and the delivery is sooo slow and
> the mail queue just is growing and growing....
>
> Nov 5 12:51:23 mailgw postfix/qmgr[14251]: warning: mail for
> [127.0.0.1]:10024 is using up 4001 of 4004 active queue entries
This is just a consequence of your amavisd+SpamAssassin not being able
to keep up with the incoming mail flow. No fine tuning on the Postfix
side will be able to compensate for the fact that your mail inflow rate
is larger than the mail processing throughput of SpamAssassin filtering.
What is your message rate on a normal day? Is the current mail flow
significantly larger? Perhaps you are under a bounce storm, which can
easily increase the mail flow rate by an order of magnitude. Examine
what kind of messages are most typical in your mail queue (mailq, postcat),
try to determine if these are just normal spam flow, or bounces, or
something else (e.g. mailer abused as an open relay, perhaps by one of
your client PCs which might have been zombiized).
What is the message throughput though the filter - see what amavisd-agent
has to report, the more interesting figures are for example:
CacheAttempts 15216 3217/h 100.0 % (CacheAttempts)
CacheHits 1750 370/h 11.5 % (CacheAttempts)
...
InMsgs 15216 3217/h 100.0 % (InMsgs)
InMsgsBounce 4176 883/h 27.4 % (InMsgs)
InMsgsBounceKilled 3904 825/h 93.5 % (InMsgsBounce)
...
TimeElapsedDecoding ...
TimeElapsedPenPals
TimeElapsedReceiving
TimeElapsedSending
TimeElapsedSpamCheck
TimeElapsedVirusCheck
TimeElapsedTotal
How does the display of amavisd-nanny look like? Are all processes
about evenly busy? Are processing times significantly longer than a
couple of seconds? Set $nanny_details_level=2; (in amavisd.conf) for
more detailed timing breakdown by amavisd-nanny.
Check timing log (at log level 2), you may want to (re)confirm that
SpamAssassin is really taking most of the time, just in case.
> -I turned off DCC, Razor and Pyzor.
> -I set the bayes use to 0.
These were pretty drastic measures, significantly affecting quality
of SA results. Once you get over the current crisis, at least put back
the DCC and Bayes on MySQL, which are relatively low resource consumers
compared to regexp-based rules and to Pyzor (razor is somewhere inbetween).
Mark