You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by "Mark T. Valites" <va...@geneseo.edu> on 2005/02/02 17:04:41 UTC
Addition Problem?
I recently installed spamassassin-3.0.2-1 from Debian testing. I've been
running it for myself and a couple other folks who are testing for me. For
the most part, I've got it working exactly like our 2.6.x scanners, except
sometimes, it seems like it just can't add.
What happens is that for some messages, it will add up the scores
incorrectly & not insert a X-Spam-Report at all & will report a score
lower than what it really is. But the kicker is that it still rewrites the
Subject header!
For example:
X-Spam-Checker-Version: SpamAssassin 3.0.2 (2004-11-16) on
myhost.mydomain.com
X-Spam-Level: **
X-Spam-Status: No, score=2.2 required=3.0 tests=DATE_IN_FUTURE_03_06,
EXTRA_MPART_TYPE,HELO_DYNAMIC_HCC,HTML_40_50,HTML_MESSAGE,
HTML_TAG_EXIST_TBODY autolearn=disabled version=3.0.2
In this case, my threshold is 3. The subject for this message is
re-written, but there is no spam report and all those tests add up to 6.19
by my count.
my 3.x local.cf has:
rewrite_header Subject *****SPAM*****
use_bayes 0
bayes_auto_learn 0
report_safe 0
and a couple custom tests for localally generated email.
The mail logs show the lower score too:
Feb 2 07:34:14 uranus spamd[281]: processing message <2e...@accesstoledo.com> for valites:16613.
Feb 2 07:34:14 uranus spamd[281]: clean message (2.2/3.0) for valites:16613 in 0.2 seconds, 4765 bytes.
Feb 2 07:34:14 uranus spamd[281]: result: . 2 - DATE_IN_FUTURE_03_06,EXTRA_MPART_TYPE,HELO_DYNAMIC_HCC,HTML_40_50,HTML_MESSAGE,HTML_TAG_EXIST_TBODY scantime=0.2,size=4765,mid=<2e...@accesstoledo.com>,autolearn=disabled
-Mark
--
Mark T. Valites
Unix Systems Analyst
Computing & Information Technology
SUNY Geneseo
>--))> >--))>
Re: Addition Problem?
Posted by jdow <jd...@earthlink.net>.
From: "Mark T. Valites" <va...@geneseo.edu>
> I recently installed spamassassin-3.0.2-1 from Debian testing. I've been
> running it for myself and a couple other folks who are testing for me. For
> the most part, I've got it working exactly like our 2.6.x scanners, except
> sometimes, it seems like it just can't add.
>
> What happens is that for some messages, it will add up the scores
> incorrectly & not insert a X-Spam-Report at all & will report a score
> lower than what it really is. But the kicker is that it still rewrites the
> Subject header!
>
> For example:
>
> X-Spam-Checker-Version: SpamAssassin 3.0.2 (2004-11-16) on
> myhost.mydomain.com
> X-Spam-Level: **
> X-Spam-Status: No, score=2.2 required=3.0 tests=DATE_IN_FUTURE_03_06,
> EXTRA_MPART_TYPE,HELO_DYNAMIC_HCC,HTML_40_50,HTML_MESSAGE,
> HTML_TAG_EXIST_TBODY autolearn=disabled version=3.0.2
>
There is a known problem with user rules. The rules work but after
the first time a given spamd spawn runs the rule scores are lost
and default to 1.
{^_^}
Re: Addition Problem?
Posted by Matt Kettler <mk...@evi-inc.com>.
At 11:33 AM 2/4/2005, Mark T. Valites wrote:
>What I'd really like to do is leave the double scanning in place & figure
>out what tests my 2.6 machines are triggering & compare those to the ones
>3.0 triggers. Is there anyway I can do this without touching the conf on
>my production 2.6 machines? I'd like to set it up where I insert the
>headers above for 3.0, but leave the X-Spam-Status and X-Spam-Report from
>the 2.6 scanning in place. clear_headers blew out the already existing 2.6
>headers.
Erm.. clear_headers should not do that. clear_headers should only clear
your header creation settings, and change nothing at all about the message
itself.
Are you sure that clear_headers stomped your ones made by the 2.6 machine?
If so, it's a very serious bug in SA 3.0.
Re: Addition Problem?
Posted by "Mark T. Valites" <va...@geneseo.edu>.
On Wed, 2 Feb 2005, Matt Kettler wrote:
> At 05:44 PM 2/2/2005, Alan Munday wrote:
> >Matt Kettler wrote the following on 02/02/2005 22:02:
> > > Actually, it's limit in SA's header generation that I forgot about.
> > > I
> > forgot that it forces X-Spam as a prefix.
> > >
> >
> >Does that mean I can have the bonus point after all?
>
> Sure.
>
> I'd offer you a cookie too, but I suspect that I'm a bit too far away
> for you to drop in :)
I also figured out the header rewriting, and I too live in the Eastern
part of the US. Is Rochester, NY too far from Maryland for a cookie? :)
You were dead on with the multiple scans. From lurking here a couple days
now now, I see it looks like a fairly common thing to overlook. In my
case, I'm using exim4 & was calling spamc from a procmail router called
*after* my spamassassin router which pointed at a different set of spamd
machines. The easiest way to verify this was to change the rewrite_header
Subject value to something slightly different - ie:
rewrite_header Subject *****SPAM-3.0*****
What I ended up with was a bunch of messages subjects that looked like
this:
Subject: *****SPAM-3.0***** *****SPAM***** <original subject>
Unfortunately, those were in the minority, and most were just:
Subject: *****SPAM***** HOW <original subject>
I had none that looked like:
Subject: *****SPAM-3.0***** <original subject>
It looks like SA-3 is catching far fewer than the 2.6.x machines!
A diff between my 2.6.x and 3.x local.cf files yields:
< rewrite_header Subject *****SPAM-3.0*****
<
< add_header all 3-Status _YESNO_, score=_SCORE_ required=_REQD_ tests=_TESTS_ autolearn=_AUTOLEARN_ version=_VERSION_
< add_header all 3-Level _STARS(*)_
< add_header all 3-Flag _YESNOCAPS_
< add_header all 3-Report _REPORT_
---
> rewrite_subject 1
What I'd really like to do is leave the double scanning in place & figure
out what tests my 2.6 machines are triggering & compare those to the ones
3.0 triggers. Is there anyway I can do this without touching the conf on
my production 2.6 machines? I'd like to set it up where I insert the
headers above for 3.0, but leave the X-Spam-Status and X-Spam-Report from
the 2.6 scanning in place. clear_headers blew out the already existing 2.6
headers.
Ideally I'd end up with headers like this:
X-Spam-Status: Yes, score=3.6 required=3.0 tests=ALL_NATURAL,CUM_SHOT,
MONEY_BACK autolearn=disabled version=3.0.2
X-Spam-Report:
* 1.7 CUM_SHOT BODY: Possible porn - Cum Shot
* 1.8 ALL_NATURAL BODY: Spam is 100% natural?!
* 0.0 MONEY_BACK BODY: Money back guarantee
X-Spam-3-Status: Yes, score=3.6 required=3.0 tests=ALL_NATURAL,CUM_SHOT,
MONEY_BACK autolearn=disabled version=3.0.2
X-Spam-3-Report:
* 1.7 CUM_SHOT BODY: Possible porn - Cum Shot
* 1.8 ALL_NATURAL BODY: Spam is 100% natural?!
* 0.0 MONEY_BACK BODY: Money back guarantee
Where the first status and report are from 2.6 and the second set from 3.0
-Mark
--
Mark T. Valites
Unix Systems Analyst
Computing & Information Technology
SUNY Geneseo
>--))> >--))>
Re: Addition Problem?
Posted by Matt Kettler <mk...@evi-inc.com>.
At 05:44 PM 2/2/2005, Alan Munday wrote:
>Matt Kettler wrote the following on 02/02/2005 22:02:
> > Actually, it's limit in SA's header generation that I forgot about. I
> forgot that it forces X-Spam as a prefix.
> >
>
>Does that mean I can have the bonus point after all?
Sure.
I'd offer you a cookie too, but I suspect that I'm a bit too far away for
you to drop in :)
(I'm eastern US, and you seem to be UK)
Re: Addition Problem?
Posted by Alan Munday <sp...@brightheadtechnology.com>.
Matt Kettler wrote the following on 02/02/2005 22:02:
>
> Actually, it's limit in SA's header generation that I forgot about. I
> forgot that it forces X-Spam as a prefix.
>
Does that mean I can have the bonus point after all?
Re: Addition Problem?
Posted by Matt Kettler <mk...@evi-inc.com>.
At 04:40 PM 2/2/2005, Alan Munday wrote:
>Yep, been there read that.....
>
>and that produces X-Spam-Myserver-Status etc... which was easy.
>
>But you originally specified X-Server1-Spam-Status which seems a little
>harder.
>
>I'm suspecting a wordo, like a typo, but bigger :-)
Actually, it's limit in SA's header generation that I forgot about. I
forgot that it forces X-Spam as a prefix.
I use MailScanner and that limit isn't present in MailScanner. I can modify
my headers to be literally anything I like, hence I do:
X-EVI-MailScanner-SpamCheck:
But I could tell it to do:
Y-Someheader-Garbage:
and Mailscanner would do it (although not having it begin with X- would be
a bad idea, I could do it)
Re: Addition Problem?
Posted by Alan Munday <sp...@brightheadtechnology.com>.
Matt Kettler wrote the following on 02/02/2005 21:18:
> At 04:09 PM 2/2/2005, Alan Munday wrote:
>
>> Matt Kettler wrote the following on 02/02/2005 17:12:
>> > Bonus points if you use the add_header feature to create a secondary
>> X-Spam-Status header that is X-Server1-Spam-Status:
>> >
>> OK Matt
>>
>> I liked the idea of this so have been looking through the config docs
>> to find the answer.....
>>
>> But I've failed to do so.
>>
>> So how do you use add_header to modify the X-Spam headers?
>
>
>
> Look at add_header in:
>
> http://spamassassin.apache.org/full/3.0.x/dist/doc/Mail_SpamAssassin_Conf.html
>
Yep, been there read that.....
and that produces X-Spam-Myserver-Status etc... which was easy.
But you originally specified X-Server1-Spam-Status which seems a little harder.
I'm suspecting a wordo, like a typo, but bigger :-)
Alan
Re: Addition Problem?
Posted by Matt Kettler <mk...@evi-inc.com>.
At 04:09 PM 2/2/2005, Alan Munday wrote:
>Matt Kettler wrote the following on 02/02/2005 17:12:
> > Bonus points if you use the add_header feature to create a secondary
> X-Spam-Status header that is X-Server1-Spam-Status:
> >
>OK Matt
>
>I liked the idea of this so have been looking through the config docs to
>find the answer.....
>
>But I've failed to do so.
>
>So how do you use add_header to modify the X-Spam headers?
Look at add_header in:
http://spamassassin.apache.org/full/3.0.x/dist/doc/Mail_SpamAssassin_Conf.html
Basically, Just duplicate the default "Status" header setting with
"Myserver-Status".
This will make SA create the default X-Spam-Status, as well as
X-Spam-Myserver-Status.
add_header all Myserver-Status _YESNO_, score=_SCORE_ required=_REQD_
tests=_TESTS_ autolearn=_AUTOLEARN_ version=_VERSION_
If you wanted to get rid of the creation of the original X-Spam-Status
header, you'd need to do a clear_headers first, and re-add all the headers
you want to still be generated:
clear_headers
add_header spam Flag _YESNOCAPS_
add_header all Myserver-Status _YESNO_, score=_SCORE_ required=_REQD_
tests=_TESTS_ autolearn=_AUTOLEARN_ version=_VERSION_
add_header all Level _STARS(*)_
add_header all Checker-Version SpamAssassin _VERSION_ (_SUBVERSION_) on
_HOSTNAME_
Re: Addition Problem?
Posted by Alan Munday <sp...@brightheadtechnology.com>.
Matt Kettler wrote the following on 02/02/2005 17:12:
>
> Bonus points if you use the add_header feature to create a secondary
> X-Spam-Status header that is X-Server1-Spam-Status:
>
OK Matt
I liked the idea of this so have been looking through the config docs to find the answer.....
But I've failed to do so.
So how do you use add_header to modify the X-Spam headers?
Thanks
Alan
And I really wanted that bonus point.
Re: Addition Problem?
Posted by Matt Kettler <mk...@evi-inc.com>.
At 11:50 AM 2/2/2005, Mark T. Valites wrote:
> > Using scoreset 1 I get I get 2.12.
>
>This still begs the question of why the subject is being re-written for
>some messages if the score is below my threshold.
It may have been scanned twice, and got marked the first time (perhaps even
by a different system in a different network). The second scan will
over-write the spam-status header, reflecting a lower score.
Try customizing your spam tags (and make them different for each server you
run if you have multiple).
Bonus points if you use the add_header feature to create a secondary
X-Spam-Status header that is X-Server1-Spam-Status:
Re: Addition Problem?
Posted by "Mark T. Valites" <va...@geneseo.edu>.
On Wed, 2 Feb 2005, Matt Kettler wrote:
> At 11:04 AM 2/2/2005, Mark T. Valites wrote:
> >X-Spam-Status: No, score=2.2 required=3.0 tests=DATE_IN_FUTURE_03_06,
> > EXTRA_MPART_TYPE,HELO_DYNAMIC_HCC,HTML_40_50,HTML_MESSAGE,
> > HTML_TAG_EXIST_TBODY autolearn=disabled version=3.0.2
> >
> >In this case, my threshold is 3. The subject for this message is
> >re-written, but there is no spam report and all those tests add up to 6.19
> >by my count.
>
>
>
> You seem to have added up the set 0 scores, but you're using set 1.
Whoops - I missed the 'net' column.
> Using scoreset 1 I get I get 2.12.
This still begs the question of why the subject is being re-written for
some messages if the score is below my threshold.
Thanks Matt.
--
Mark T. Valites
Unix Systems Analyst
Computing & Information Technology
SUNY Geneseo
>--))> >--))>
Re: Addition Problem?
Posted by Matt Kettler <mk...@evi-inc.com>.
At 11:04 AM 2/2/2005, Mark T. Valites wrote:
>X-Spam-Status: No, score=2.2 required=3.0 tests=DATE_IN_FUTURE_03_06,
> EXTRA_MPART_TYPE,HELO_DYNAMIC_HCC,HTML_40_50,HTML_MESSAGE,
> HTML_TAG_EXIST_TBODY autolearn=disabled version=3.0.2
>
>In this case, my threshold is 3. The subject for this message is
>re-written, but there is no spam report and all those tests add up to 6.19
>by my count.
You seem to have added up the set 0 scores, but you're using set 1.
Using scoreset 1 I get I get 2.12.
Since EXTRA_MPART_TYPE matched, you must be using the set 1 score column
(the second number), as the others contain 0's and the rule would not
otherwise match.
This indicates SA is using scores for net-checks enabled with bayes checks
disabled
-------------------------------------
score DATE_IN_FUTURE_03_06 1.288 0.072 2.052 0.847
score EXTRA_MPART_TYPE 0 0.222 0 0
score HELO_DYNAMIC_HCC 3.360 1.540 2.451 3.741
score HTML_40_50 0.527 0.086 0.052 0.035
score HTML_MESSAGE 0.001
score HTML_TAG_EXIST_TBODY 1.014 0.233 0.079 0.114
0.072+0.222+1.540+0.052+.001+0.233= 2.120