You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Pete Dubler <pe...@dublerfamily.com> on 2005/11/10 00:36:33 UTC
some mail still not getting X-SPAM headers
I have been running spamassassin for many years with great success.
Recently though I noticed more spam getting through so I updated to
SA 3.1 on my Redhat 9 system. I also added some additional cf's for SA
using ruledujour in hopes of catching more spam. Now however, lots of
messages never seem to get scanned by spamc. The X-SPAM headers
do not appear on SOME messages but it works fine for the majority. (of
course
those getting through are usually spam)... YES, I have read all of the
FAQs and posting I could find on this issue and I still cannot seem to
solve the problem. I have added a bunch of diagnostic log messages to
my /etc/procmailrc to show me when mail goes to spamc and if/when it returns
(see below). I have also tried some tweeks to the SPAMOPTIONS in the
/etc/init.d/spamassassin file in an effort to make sure I had plenty of
children available and that the children were being killed and reborn
more often than the default (200). It does not seem to matter how many
messages
are received by the system at any one time. Even a single message every
10 minutes can
demonstrate this problem of some messages not getting X-SPAM headers.
I have been working on this for quite a
while now and am making no progress toward improving the problem. Below
is information
on the spamdoptions, my invocation of spamc, and log files from
successful passes through
spamc and unsuccessful passes.
Any ideas to solve this problem would be greatly appreciated.
Other data:
- I used cpan to update from 2.x to 3.1
- other cf's being used in spamassassin (ls of /etc/mail/spamassassin):
70_sare_adult.cf
70_sare_highrisk.cf
99_sare_fraud_post25x.cf
antidrug.cf
blacklist.cf
blacklist-uri.cf
init.pre
local.cf
local.cf.extensions
local.cf.old
local.cf.old2
local.cf.rpmnew
local.cf.rpmsave
random.cf
tripwire.cf
v310.pre
OTHER INFORMATION:
When I run
service spamassassin restart
I often get this back:
Shutting down spamd: /etc/init.d/spamassassin: line 201: kill: (1129) -
No such
process
/etc/init.d/spamassassin: line 201: kill: (1122) - No such process
/etc/init.d/spamassassin: line 201: kill: (32205) - No such process
/etc/init.d/spamassassin: line 201: kill: (31062) - No such process
[ OK ]
Starting spamd: [ OK ]
*******************************************************
Options I am using to start spamd:
SPAMDOPTIONS="-d -c -m10 --ident-timeout=15 --max-conn-per-child=75
--timeout-tcp=90"
*******************************************************
Invocation of spamc (rc.spam)
# send mail through spamassassin
DROPPRIVS=yes
LOG="FIRST PASS THROUGH SPAMC
"
:0fw
* < 256000
|/usr/bin/spamc
LOG="`/bin/date` FIRST PASS COMPLETED
"
# Try a second time if SpamAssassin failed
# this time, wait a random amount of time before starting
LOG="CHECKING IF SPAMC COMPLETED
"
:0fw
* ! ^X-Spam-Level:.*
* < 256000
{
LOG="`/bin/date` **** SPAMC FAILED. RETRYING ****
"
# Here we will sleep at least 15 second but less than 30 seconds
# then attempt to run spamc again
| sleep $(perl -le 'srand($$);print int(rand()*15)+15'); /usr/bin/spamc
LOG="`bin/date` BACK FROM SECOND ATTEMPT AT SPAMC
"
}
# Mail with score of 15 or higher as almost certainly spam
:0:
* ^X-Spam-Level: \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*
/var/spool/mail/spam_15_points_or_more
# Work around procmail bug: any output on stderr will cause the
# "F" in "From" to be dropped. This will re-add it.
:0
* ^^rom[ ]
{
LOG="*** Dropped F off From_ header! Fixing up. "
:0 fhw
| sed -e 'ls/^/F/'
}
# each user's procmail rc calls a custom rc.spam that files spam in
their own
# spam file under /var/spool/mail
*******************************************************
example of log when things are working:
#######&&&&&&&& NEW MESSAGE &&&&&&&&&&#########
Wed Nov 9 00:12:09 MST 2005 *** going to vsnag.rc
Wed Nov 9 00:12:09 MST 2005 **** back from vsnag.rc
Wed Nov 9 00:12:10 MST 2005 ***** back from npd.rc
Wed Nov 9 00:12:10 MST 2005 CALLING RC.SPAM
FIRST PASS THROUGH SPAMC
Wed Nov 9 00:22:10 MST 2005 FIRST PASS COMPLETED
CHECKING IF SPAMC COMPLETED
procmail: Extraneous filter-flag ignored
Wed Nov 9 00:22:10 MST 2005 **** SPAMC FAILED. RETRYING ****
procmail: Skipped "| sleep $(perl -le 'srand($$);print
int(rand()*15)+15'); /usr/bin/spamc"
bin/date: bin/date: No such file or directory
BACK FROM SECOND ATTEMPT AT SPAMC
Wed Nov 9 00:22:10 MST 2005 BACK FROM RC.SPAM
*******************************************************
example of log when things are NOT working:
#######&&&&&&&& NEW MESSAGE &&&&&&&&&&#########
Wed Nov 9 01:06:05 MST 2005 *** going to vsnag.rc
Wed Nov 9 01:06:05 MST 2005 **** back from vsnag.rc
Wed Nov 9 01:06:05 MST 2005 ***** back from npd.rc
Wed Nov 9 01:06:05 MST 2005 CALLING RC.SPAM
FIRST PASS THROUGH SPAMC
*******************************************************
as you can see, when things are not working, the mail goes to spamc but
never comes out of the process. (never gets to LOG="FIRST PASS COMPLETED")
mail does however drop through to email box with no X-SPAM... headers.
Re: some mail still not getting X-SPAM headers
Posted by Pete Dubler <pe...@dublerfamily.com>.
Matt,
Thanks for the prompt reply. Indeed you ask a very good question.
No the size of the message seems to not matter. When I turn on the
expanded logging I can see that the message is in fact "assigned" to the
spamc process.
Other ideas?
Pete
Matt Kettler wrote:
>Question,
>
>Are the "failed" messages > 256000 bytes in size?
>
>If so, you procmail rules are bypassing the calls to spamc. Hence you see the
>"first pass" message, but never a pass/fail message afterwards.
>
>
>
>
>># send mail through spamassassin
>>DROPPRIVS=yes
>>LOG="FIRST PASS THROUGH SPAMC
>>"
>>:0fw
>>* < 256000
>>|/usr/bin/spamc
>>
>>LOG="`/bin/date` FIRST PASS COMPLETED
>>"
>># Try a second time if SpamAssassin failed
>># this time, wait a random amount of time before starting
>>LOG="CHECKING IF SPAMC COMPLETED
>>"
>>:0fw
>>* ! ^X-Spam-Level:.*
>>* < 256000
>>{
>>LOG="`/bin/date` **** SPAMC FAILED. RETRYING ****
>>"
>># Here we will sleep at least 15 second but less than 30 seconds
>># then attempt to run spamc again
>>| sleep $(perl -le 'srand($$);print int(rand()*15)+15'); /usr/bin/spamc
>>LOG="`bin/date` BACK FROM SECOND ATTEMPT AT SPAMC
>>"
>>}
>>
>># Mail with score of 15 or higher as almost certainly spam
>>:0:
>>* ^X-Spam-Level: \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*
>>/var/spool/mail/spam_15_points_or_more
>>
>># Work around procmail bug: any output on stderr will cause the
>># "F" in "From" to be dropped. This will re-add it.
>>:0
>>* ^^rom[ ]
>>{
>> LOG="*** Dropped F off From_ header! Fixing up. "
>>
>> :0 fhw
>> | sed -e 'ls/^/F/'
>>}
>>
>># each user's procmail rc calls a custom rc.spam that files spam in
>>their own
>># spam file under /var/spool/mail
>>
>>
>>
>
><snip>
>
>
>>*******************************************************
>>example of log when things are NOT working:
>>#######&&&&&&&& NEW MESSAGE &&&&&&&&&&#########
>>Wed Nov 9 01:06:05 MST 2005 *** going to vsnag.rc
>>Wed Nov 9 01:06:05 MST 2005 **** back from vsnag.rc
>>Wed Nov 9 01:06:05 MST 2005 ***** back from npd.rc
>>Wed Nov 9 01:06:05 MST 2005 CALLING RC.SPAM
>>FIRST PASS THROUGH SPAMC
>>
>>*******************************************************
>>as you can see, when things are not working, the mail goes to spamc but
>>never comes out of the process. (never gets to LOG="FIRST PASS COMPLETED")
>>mail does however drop through to email box with no X-SPAM... headers.
>>
>>
>
>Are you sure? It's possible you never get into spamc at all, since your log
>"FIRST PASS THROUGH SPAMC" is unconditional, but your call to spamc might get
>bypassed.
>
>
>
>
>
Re: some mail still not getting X-SPAM headers
Posted by Pete Dubler <pe...@dublerfamily.com>.
Matt,
your email in fact pointed out a bit of sloopiness on my part in the
logging. I have changed
it so it now should show correctly which messages actually trigger a
call to spamc without
me having to look at the verbose log listings.
Thanks again,
Pete
HERE IS THE NEW rc.spam
# send mail through spamassassin
DROPPRIVS=yes
LOG="TESTING FOR SIZE B4 SENDING TO SPAMC"
"
:0fw
* < 256000
{
LOG="/`/bin/date` FIRST PASS THROUGH SPAMC
"
|/usr/bin/spamc
LOG="`/bin/date` FIRST PASS COMPLETED
"
}
# Try a second time if SpamAssassin failed
# this time, wait a random amount of time before starting
LOG="CHECKING IF SPAMC COMPLETED
"
:0fw
* ! ^X-Spam-Level:.*
* < 256000
{
LOG="`/bin/date` **** SPAMC FAILED. RETRYING ****
"
# Here we will sleep at least 15 second but less than 30 seconds
# then attempt to run spamc again
| sleep $(perl -le 'srand($$);print int(rand()*15)+15'); /usr/bin/spamc
LOG="`bin/date` BACK FROM SECOND ATTEMPT AT SPAMC
"
}
# Mail with score of 15 or higher as almost certainly spam
:0:
* ^X-Spam-Level: \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*
/var/spool/mail/spam_15_points_or_more
# Work around procmail bug: any output on stderr will cause the
# "F" in "From" to be dropped. This will re-add it.
:0
* ^^rom[ ]
{
LOG="*** Dropped F off From_ header! Fixing up. "
:0 fhw
| sed -e 'ls/^/F/'
}
# each user's procmail rc calls a custom rc.spam that files spam in
their own
# spam file under /var/spool/mail
Matt Kettler wrote:
>Question,
>
>Are the "failed" messages > 256000 bytes in size?
>
>If so, you procmail rules are bypassing the calls to spamc. Hence you see the
>"first pass" message, but never a pass/fail message afterwards.
>
>
>
>
>># send mail through spamassassin
>>DROPPRIVS=yes
>>LOG="FIRST PASS THROUGH SPAMC
>>"
>>:0fw
>>* < 256000
>>|/usr/bin/spamc
>>
>>LOG="`/bin/date` FIRST PASS COMPLETED
>>"
>># Try a second time if SpamAssassin failed
>># this time, wait a random amount of time before starting
>>LOG="CHECKING IF SPAMC COMPLETED
>>"
>>:0fw
>>* ! ^X-Spam-Level:.*
>>* < 256000
>>{
>>LOG="`/bin/date` **** SPAMC FAILED. RETRYING ****
>>"
>># Here we will sleep at least 15 second but less than 30 seconds
>># then attempt to run spamc again
>>| sleep $(perl -le 'srand($$);print int(rand()*15)+15'); /usr/bin/spamc
>>LOG="`bin/date` BACK FROM SECOND ATTEMPT AT SPAMC
>>"
>>}
>>
>># Mail with score of 15 or higher as almost certainly spam
>>:0:
>>* ^X-Spam-Level: \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*
>>/var/spool/mail/spam_15_points_or_more
>>
>># Work around procmail bug: any output on stderr will cause the
>># "F" in "From" to be dropped. This will re-add it.
>>:0
>>* ^^rom[ ]
>>{
>> LOG="*** Dropped F off From_ header! Fixing up. "
>>
>> :0 fhw
>> | sed -e 'ls/^/F/'
>>}
>>
>># each user's procmail rc calls a custom rc.spam that files spam in
>>their own
>># spam file under /var/spool/mail
>>
>>
>>
>
><snip>
>
>
>>*******************************************************
>>example of log when things are NOT working:
>>#######&&&&&&&& NEW MESSAGE &&&&&&&&&&#########
>>Wed Nov 9 01:06:05 MST 2005 *** going to vsnag.rc
>>Wed Nov 9 01:06:05 MST 2005 **** back from vsnag.rc
>>Wed Nov 9 01:06:05 MST 2005 ***** back from npd.rc
>>Wed Nov 9 01:06:05 MST 2005 CALLING RC.SPAM
>>FIRST PASS THROUGH SPAMC
>>
>>*******************************************************
>>as you can see, when things are not working, the mail goes to spamc but
>>never comes out of the process. (never gets to LOG="FIRST PASS COMPLETED")
>>mail does however drop through to email box with no X-SPAM... headers.
>>
>>
>
>Are you sure? It's possible you never get into spamc at all, since your log
>"FIRST PASS THROUGH SPAMC" is unconditional, but your call to spamc might get
>bypassed.
>
>
>
>
>
Re: some mail still not getting X-SPAM headers
Posted by Matt Kettler <mk...@evi-inc.com>.
Question,
Are the "failed" messages > 256000 bytes in size?
If so, you procmail rules are bypassing the calls to spamc. Hence you see the
"first pass" message, but never a pass/fail message afterwards.
> # send mail through spamassassin
> DROPPRIVS=yes
> LOG="FIRST PASS THROUGH SPAMC
> "
> :0fw
> * < 256000
> |/usr/bin/spamc
>
> LOG="`/bin/date` FIRST PASS COMPLETED
> "
> # Try a second time if SpamAssassin failed
> # this time, wait a random amount of time before starting
> LOG="CHECKING IF SPAMC COMPLETED
> "
> :0fw
> * ! ^X-Spam-Level:.*
> * < 256000
> {
> LOG="`/bin/date` **** SPAMC FAILED. RETRYING ****
> "
> # Here we will sleep at least 15 second but less than 30 seconds
> # then attempt to run spamc again
> | sleep $(perl -le 'srand($$);print int(rand()*15)+15'); /usr/bin/spamc
> LOG="`bin/date` BACK FROM SECOND ATTEMPT AT SPAMC
> "
> }
>
> # Mail with score of 15 or higher as almost certainly spam
> :0:
> * ^X-Spam-Level: \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*
> /var/spool/mail/spam_15_points_or_more
>
> # Work around procmail bug: any output on stderr will cause the
> # "F" in "From" to be dropped. This will re-add it.
> :0
> * ^^rom[ ]
> {
> LOG="*** Dropped F off From_ header! Fixing up. "
>
> :0 fhw
> | sed -e 'ls/^/F/'
> }
>
> # each user's procmail rc calls a custom rc.spam that files spam in
> their own
> # spam file under /var/spool/mail
>
<snip>
> *******************************************************
> example of log when things are NOT working:
> #######&&&&&&&& NEW MESSAGE &&&&&&&&&&#########
> Wed Nov 9 01:06:05 MST 2005 *** going to vsnag.rc
> Wed Nov 9 01:06:05 MST 2005 **** back from vsnag.rc
> Wed Nov 9 01:06:05 MST 2005 ***** back from npd.rc
> Wed Nov 9 01:06:05 MST 2005 CALLING RC.SPAM
> FIRST PASS THROUGH SPAMC
>
> *******************************************************
> as you can see, when things are not working, the mail goes to spamc but
> never comes out of the process. (never gets to LOG="FIRST PASS COMPLETED")
> mail does however drop through to email box with no X-SPAM... headers.
Are you sure? It's possible you never get into spamc at all, since your log
"FIRST PASS THROUGH SPAMC" is unconditional, but your call to spamc might get
bypassed.