You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spamassassin.apache.org by Justin Mason <jm...@jmason.org> on 2011/02/21 18:23:57 UTC
mass-check using network checks with --net?
hi all --
I've been having some problems with the SOUGHT generation backend --
it appears current trunk's "mass-check" is taking a very long time
(ie. multiple days for 20k messages), without the --net switch. I
noticed some warnings previously which referred to the AskDNS plugin.
is there a possibility this is accessing DNS on each message even
without --net?
--j.
Re: mass-check using network checks with --net?
Posted by Mark Martinec <Ma...@ijs.si>.
> fwiw, removing 70_sandbox.cf hasn't had too much effect on that, so
> it's a promoted rule in 72_active.cf.
loadplugin HitFreqsRuleTiming masses/plugins/HitFreqsRuleTiming.pm
results in: ./timing.log
Mark
Re: mass-check using network checks with --net?
Posted by Justin Mason <jm...@jmason.org>.
fwiw, removing 70_sandbox.cf hasn't had too much effect on that, so
it's a promoted rule in 72_active.cf.
--j.
On Tue, Feb 22, 2011 at 11:07, Justin Mason <jm...@jmason.org> wrote:
> Feb 22 11:07:10.729 [32583] dbg: timing: total 6020 ms - parse: 3
> (0.0%), extract_message_metadata: 43 (0.7%), get_uri_detail_list: 81
> (1.4%), tests_pri_-1000: 56 (0.9%), tests_pri_-950: 1.72 (0.0%),
> tests_pri_-900: 38 (0.6%), tests_pri_-400: 1.68 (0.0%), tests_pri_0:
> 5664 (94.1%), tests_pri_500: 94 (1.6%)
>
> so it's tests at pri 0. doesn't really narrow it down too much ;)
>
> --j.
>
> On Tue, Feb 22, 2011 at 00:16, Mark Martinec <Ma...@ijs.si> wrote:
>> Justin wrote:
>>> unfortunately I didn't save them; just "svn up"d and restarted the
>>> process to see if it fixed it (it didn't, but the messages are no
>>> more).
>>> some strace'ing isn't showing network traffic, so it could just be
>>> poor local rule performance with the current trunk ruleset.
>>> Warren: Ubuntu 10.04, default /usr/bin/perl.
>>
>> Try: mass-check --debug=timing
>> to see if anything unusual stands out.
>>
>> (after first updating svn - I just added a call to timer_reset
>> to mass-check before processing each message, like the spamd does;
>> previously the timing report would show cumulative times, which are
>> not that interesting)
>>
>> Mark
>>
>
Re: mass-check using network checks with --net?
Posted by Justin Mason <jm...@jmason.org>.
Feb 22 11:07:10.729 [32583] dbg: timing: total 6020 ms - parse: 3
(0.0%), extract_message_metadata: 43 (0.7%), get_uri_detail_list: 81
(1.4%), tests_pri_-1000: 56 (0.9%), tests_pri_-950: 1.72 (0.0%),
tests_pri_-900: 38 (0.6%), tests_pri_-400: 1.68 (0.0%), tests_pri_0:
5664 (94.1%), tests_pri_500: 94 (1.6%)
so it's tests at pri 0. doesn't really narrow it down too much ;)
--j.
On Tue, Feb 22, 2011 at 00:16, Mark Martinec <Ma...@ijs.si> wrote:
> Justin wrote:
>> unfortunately I didn't save them; just "svn up"d and restarted the
>> process to see if it fixed it (it didn't, but the messages are no
>> more).
>> some strace'ing isn't showing network traffic, so it could just be
>> poor local rule performance with the current trunk ruleset.
>> Warren: Ubuntu 10.04, default /usr/bin/perl.
>
> Try: mass-check --debug=timing
> to see if anything unusual stands out.
>
> (after first updating svn - I just added a call to timer_reset
> to mass-check before processing each message, like the spamd does;
> previously the timing report would show cumulative times, which are
> not that interesting)
>
> Mark
>
Re: mass-check using network checks with --net?
Posted by Mark Martinec <Ma...@ijs.si>.
Justin wrote:
> unfortunately I didn't save them; just "svn up"d and restarted the
> process to see if it fixed it (it didn't, but the messages are no
> more).
> some strace'ing isn't showing network traffic, so it could just be
> poor local rule performance with the current trunk ruleset.
> Warren: Ubuntu 10.04, default /usr/bin/perl.
Try: mass-check --debug=timing
to see if anything unusual stands out.
(after first updating svn - I just added a call to timer_reset
to mass-check before processing each message, like the spamd does;
previously the timing report would show cumulative times, which are
not that interesting)
Mark
Re: mass-check using network checks with --net?
Posted by Justin Mason <jm...@jmason.org>.
On Mon, Feb 21, 2011 at 18:32, Mark Martinec <Ma...@ijs.si> wrote:
>> > I've been having some problems with the SOUGHT generation backend --
>> > it appears current trunk's "mass-check" is taking a very long time
>> > (ie. multiple days for 20k messages), without the --net switch. I
>> > noticed some warnings previously which referred to the AskDNS plugin.
>> > is there a possibility this is accessing DNS on each message even
>> > without --net?
>>
>> Possibly (not intentionally). Checking...
>
> Actually - not likely.
>
> sub extract_metadata {
> [...]
> return if !$pms->is_dns_available;
>
>
> The -L option on spamassassin turns off all AskDSN activities
> according to a debug log. Didn't try it with mass-check.
>
> What warnings were you receiving?
unfortunately I didn't save them; just "svn up"d and restarted the
process to see if it fixed it (it didn't, but the messages are no
more).
some strace'ing isn't showing network traffic, so it could just be
poor local rule performance with the current trunk ruleset.
Warren: Ubuntu 10.04, default /usr/bin/perl.
--j.
Re: mass-check using network checks with --net?
Posted by Mark Martinec <Ma...@ijs.si>.
> > I've been having some problems with the SOUGHT generation backend --
> > it appears current trunk's "mass-check" is taking a very long time
> > (ie. multiple days for 20k messages), without the --net switch. I
> > noticed some warnings previously which referred to the AskDNS plugin.
> > is there a possibility this is accessing DNS on each message even
> > without --net?
>
> Possibly (not intentionally). Checking...
Actually - not likely.
sub extract_metadata {
[...]
return if !$pms->is_dns_available;
The -L option on spamassassin turns off all AskDSN activities
according to a debug log. Didn't try it with mass-check.
What warnings were you receiving?
Mark
Re: mass-check using network checks with --net?
Posted by Mark Martinec <Ma...@ijs.si>.
> I've been having some problems with the SOUGHT generation backend --
> it appears current trunk's "mass-check" is taking a very long time
> (ie. multiple days for 20k messages), without the --net switch. I
> noticed some warnings previously which referred to the AskDNS plugin.
> is there a possibility this is accessing DNS on each message even
> without --net?
Possibly (not intentionally). Checking...
Mark
Re: mass-check using network checks with --net?
Posted by "Warren Togami Jr." <wt...@gmail.com>.
On 2/21/2011 7:23 AM, Justin Mason wrote:
> hi all --
>
> I've been having some problems with the SOUGHT generation backend --
> it appears current trunk's "mass-check" is taking a very long time
> (ie. multiple days for 20k messages), without the --net switch. I
> noticed some warnings previously which referred to the AskDNS plugin.
> is there a possibility this is accessing DNS on each message even
> without --net?
>
> --j.
Do you know when this began?
What distro and perl version do you have?
Warren