You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spamassassin.apache.org by Justin Mason <jm...@jmason.org> on 2008/03/25 21:18:14 UTC
Re: distributed mass-checks on zone looping again (fwd)
Daryl C. W. O'Shea writes:
> Might be a problem if *all* of the messages are inaccessible. I don't
> think I have ever tested that scenario.
hmm. a good deal of messages should be fine! I guess the date limits
could exclude them, though--- maybe...
> Otherwise, I know cs_max_tries will limit the number of retries to the
> default of 3... I get a few perms denied errors in my mass-check logs
> every now and again (which is why I implemented cs_max_tries).
>
> I'll take a look at the full log later on.
cool. for what it's worth I found a block of NUL bytes in the middle
of the file, which looks suspicious... I've ensured every process on
the zone was killed, then restarted them; it might be worth seeing
what happens tomorrow.
--j.
> Daryl
>
>
> On 25/03/2008 12:28 PM, Justin Mason wrote:
> > is this caused by the "Permission denied" errors?
> >
> >
> > : exit=0 Tue Mar 25 16:16:32 GMT 2008; cd /home/bbmass/mc-nightly/jm
> > : jm 341...; ls -la corpus-nightly.log
> > -rw-r--r-- 1 bbmass automc 7610008 Mar 25 16:14 corpus-nightly.log
> >
> > Using corpus-nightly settings from
> > /export/home/bbmass/mc-nightly/jm/.corpus at
> > Tue Mar 25 09:11:11 GMT 2008
> > + renice -n +10 -p 13810
> > [...]
> >
> > + /local/perl586/bin/perl mass-check --noisy --progress --cs_ssl --server spamassassin.zones.apache.org.:38891 '--run_post_scan=./rule-qa/nightly-slaves-start mccli@vmsa.apache.org jm@talon1.pccc.com:-j16 jm@infiltrator.stdlib.net:-j16' --reuse --cache --cachedir=/tmpfs/aicache_nightly --cs_schedule_cache --cs_cachedir=/export/home/bbmass/cache --restart=500 'ham:detect:/home/bbmass/rawcor/jm/ham/*' --after=-15552000 --tail=40000 --scanprob=0.3 'spam:detect:/home/bbmass/rawcor/jm/spam/*'
> > **************************************************************************
> > * *
> > * Corpus will be run un-sorted but with date stamp loggging which is *
> > * needed for score generation log selection but is not available when *
> > * using the -n option. This may affect the results of any bayes and *
> > * AWL tests run during this mass-check. *
> > * *
> > **************************************************************************
> > netset: cannot include 127.0.0.0/8 as it has already been included
> > netset: cannot include 127.0.0.0/8 as it has already been included
> > status: starting scan stage now: 2008-03-25 09:12:11
> > ............................................................
> > ............................................................
> > ............................................................
> > ............................................................
> > .................................archive-iterator: unable to open /home/bbmass/rawcor/jm/ham/pub.radish.200801160000/13: Permission denied
> > archive-iterator: unable to open /home/bbmass/rawcor/jm/ham/pub.radish.200801160000/19: Permission denied
> > archive-iterator: unable to open /home/bbmass/rawcor/jm/ham/pub.radish.200801160000/21: Permission denied
> > archive-iterator: unable to open /home/bbmass/rawcor/jm/ham/pub.radish.200801160000/7: Permission denied
> >
> > [...]
> > status: POST request from 85.91.5.16 now: 2008-03-25 16:18:56
> > status: client requested 100 messages now: 2008-03-25 16:18:56
> > status: 0 messages outstanding now: 2008-03-25 16:18:58
> > status: 0 messages outstanding now: 2008-03-25 16:18:58
> > status: POST request from 140.211.11.138 now: 2008-03-25 16:18:58
> > status: client requested 100 messages now: 2008-03-25 16:18:58
> > status: 0 messages outstanding now: 2008-03-25 16:19:00
> > status: 0 messages outstanding now: 2008-03-25 16:20:45
> > status: POST request from 63.216.184.50 now: 2008-03-25 16:20:45
> > status: client requested 100 messages now: 2008-03-25 16:20:45
> > status: 0 messages outstanding now: 2008-03-25 16:20:47
> > status: 0 messages outstanding now: 2008-03-25 16:20:59
> > status: POST request from 85.91.5.16 now: 2008-03-25 16:20:59
> > status: client requested 100 messages now: 2008-03-25 16:20:59
> > status: 0 messages outstanding now: 2008-03-25 16:21:01
> > status: 0 messages outstanding now: 2008-03-25 16:21:01
> > status: POST request from 140.211.11.138 now: 2008-03-25 16:21:01
> > status: client requested 100 messages now: 2008-03-25 16:21:01
> > status: 0 messages outstanding now: 2008-03-25 16:21:02
> > status: 0 messages outstanding now: 2008-03-25 16:22:48
> > status: POST request from 63.216.184.50 now: 2008-03-25 16:22:48
> > status: client requested 100 messages now: 2008-03-25 16:22:48
> > status: 0 messages outstanding now: 2008-03-25 16:23:21
> > status: 0 messages outstanding now: 2008-03-25 16:23:22
> > status: 0 messages outstanding now: 2008-03-25 16:23:22
> > status: POST request from 140.211.11.138 now: 2008-03-25 16:23:22
> > status: client requested 100 messages now: 2008-03-25 16:23:22
> > status: 0 messages outstanding now: 2008-03-25 16:23:25
> > status: POST request from 85.91.5.16 now: 2008-03-25 16:23:25
> > status: client requested 100 messages now: 2008-03-25 16:23:25
> > status: 0 messages outstanding now: 2008-03-25 16:23:27
> > status: 0 messages outstanding now: 2008-03-25 16:25:22
> > status: POST request from 63.216.184.50 now: 2008-03-25 16:25:22
> > status: client requested 100 messages now: 2008-03-25 16:25:22
> > status: 0 messages outstanding now: 2008-03-25 16:25:28
> > status: 0 messages outstanding now: 2008-03-25 16:25:28
> > status: 0 messages outstanding now: 2008-03-25 16:25:29
> > status: POST request from 140.211.11.138 now: 2008-03-25 16:25:29
> > status: client requested 100 messages now: 2008-03-25 16:25:29
> > status: 0 messages outstanding now: 2008-03-25 16:25:35
> > status: POST request from 85.91.5.16 now: 2008-03-25 16:25:35
> > status: client requested 100 messages now: 2008-03-25 16:25:35
> > status: 0 messages outstanding now: 2008-03-25 16:25:37
> >
Re: distributed mass-checks on zone looping again (fwd)
Posted by "Daryl C. W. O'Shea" <sp...@dostech.ca>.
On 25/03/2008 4:18 PM, Justin Mason wrote:
> Daryl C. W. O'Shea writes:
>> I'll take a look at the full log later on.
>
> cool. for what it's worth I found a block of NUL bytes in the middle
> of the file, which looks suspicious... I've ensured every process on
> the zone was killed, then restarted them; it might be worth seeing
> what happens tomorrow.
Yeah, bit of a mess. Two instances of it were running at the same time,
"Could not create socket: Address already in use"... we should probably
avoid that.
Daryl
Re: distributed mass-checks on zone looping again (fwd)
Posted by "Daryl C. W. O'Shea" <sp...@dostech.ca>.
On 25/03/2008 4:18 PM, Justin Mason wrote:
> Daryl C. W. O'Shea writes:
>> Might be a problem if *all* of the messages are inaccessible. I don't
>> think I have ever tested that scenario.
>
> hmm. a good deal of messages should be fine! I guess the date limits
> could exclude them, though--- maybe...
OK, it has got 14k+ messages, so that's OK. I do think it's a bug being
triggered by messages that become inaccessible or exceed the retry count
though. A quick look doesn't yield anything obvious that I missed in
the code (which was shoe-horned into the existing logic), so I'll try to
make some time to look at it more later today.
On a possibly related note (it might be why we're seeing the bug being
triggered but not the cause of the bug), there are a small bunch of
these that look un-good:
plugin: eval failed: panic: unknown regstclass 0 at
/home/jm/nightlymc_jm/code/masses/../rules/70_sandbox.cf, rule
T_FU_LONG_HTML, line 1.
Daryl