You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Michael D Schleif <md...@helices.org> on 2004/02/25 18:16:58 UTC
AWL vs. mailing lists
How are you all handling AWL and mailing lists?
# spamassassin -V
SpamAssassin version 2.63
I subscribe to over one hundred (100) mailing lists. procmail processes
every incoming message <256kB through SA, unless procmail dumps it
first.
High volume mailing lists (e.g., debian-user) have an extremely high
incidence of ham; therefore, AWL is extremely high for such lists. When
the virus/worm-of-the-day spams the list, SA identifies these as spam,
then transforms them into ham -- solely due to AWL skewing ;<
I know that I can turn OFF AWL.
Is that my only alternative?
What do you think?
--
Best Regards,
mds
mds resource
877.596.8237
-
Dare to fix things before they break . . .
-
Our capacity for understanding is inversely proportional to how much
we think we know. The more I know, the more I know I don't know . . .
--
Re: AWL vs. mailing lists
Posted by "Keith C. Ivey" <kc...@cpcug.org>.
Michael D Schleif <md...@helices.org> wrote:
> I posted those examples, such as they are, as evidence that I really get
> AWL scores that high -- since some skeptics argued that that is not
> possible. In fact, so far, everybody has accepted my examples as
> examples of high AWL scores. To now argue that they are inadequate
> examples of else, is arguably perverse ;>
Yes, I believe that you have gotten AWL adjustments of -25, but
they're irrelevant to the problem. They're not causing spam to
be miscategorized, so those high AWL adjustments don't matter.
They're a natural side effect of high scores.
You haven't posted the details of cases where AWL has caused
miscategorization, but I don't believe that the AWL adjustment
in those cases is anything like -25.
--
Keith C. Ivey <kc...@cpcug.org>
Washington, DC
Re: AWL vs. mailing lists
Posted by "Keith C. Ivey" <kc...@cpcug.org>.
Michael D Schleif <md...@helices.org> wrote:
> * "Keith C. Ivey" <kc...@cpcug.org> [2004:02:25:23:55:15-0500] scribed:
> <snip />
>
> > Along with those five examples of correctly identified spams,
> > you did post two examples where the AWL adjustment may have
> > caused a false negative. But in both cases the message
> > triggered BAYES_00, which probably had a larger contribution to
> > the miscategorization. Judging by those I'd say the real
> > problem is spam being incorrectly autolearned as ham. Spam
> > should not be getting BAYES_00 or BAYES_01, as it is in the
> > majority of your examples. Maybe you should be blaming
> > autolearning rather than autowhitelisting.
>
> I am intrigued by your last sentence. Care to expound?
It seems an obvious conclusion from the preceding sentences in
the paragraph. When spam is getting BAYES_00 or BAYES_01, it's
a strong indication that some spam has been mislearned as ham.
Usually such mislearning happens because of autolearning,
though it could also happen by mistakes in manual learning.
--
Keith C. Ivey <kc...@cpcug.org>
Washington, DC
Re: AWL vs. mailing lists
Posted by Michael D Schleif <md...@helices.org>.
* "Keith C. Ivey" <kc...@cpcug.org> [2004:02:25:23:55:15-0500] scribed:
<snip />
> Along with those five examples of correctly identified spams,
> you did post two examples where the AWL adjustment may have
> caused a false negative. But in both cases the message
> triggered BAYES_00, which probably had a larger contribution to
> the miscategorization. Judging by those I'd say the real
> problem is spam being incorrectly autolearned as ham. Spam
> should not be getting BAYES_00 or BAYES_01, as it is in the
> majority of your examples. Maybe you should be blaming
> autolearning rather than autowhitelisting.
I am intrigued by your last sentence. Care to expound?
--
Best Regards,
mds
mds resource
877.596.8237
-
Dare to fix things before they break . . .
-
Our capacity for understanding is inversely proportional to how much
we think we know. The more I know, the more I know I don't know . . .
--
Re: AWL vs. mailing lists
Posted by Michael D Schleif <md...@helices.org>.
* "Keith C. Ivey" <kc...@cpcug.org> [2004:02:25:23:55:15-0500] scribed:
> Michael D Schleif <md...@helices.org> wrote:
>
> > The problem is in the scores that I cannot present -- those that I have
> > -R'd and redisposed -- as I said. Those did *NOT* use
> > MDS_Remove_Subject; but, were from same mailing lists. They *ARE* spam;
> > but, scored as ham.
>
> They also wouldn't have had an adjustment of -25, which is what
> people have been surprised by in your description of your
> situation. The -25 results from the very high spam score
> caused by the 50-point rule. Without it, the adjustment would
> be much less -- though of course it will certainly sometimes
> cause messages' scores to be moved from spam range to the
> nonspam range.
I wish that I had saved the ideal examples. However, once I do
`spamassassin -R', and reprocess those messages, those examples are
gone. After the fact, I decided to post to this list. You have seen
the best examples that I have archived. MDS_Remove_Subject originally
scored +5, then 10, 20 and now 50, because these _obvious_, cut-and-dry
culprits should never be mistaken for ham -- never.
> > Nevertheless, skewing my point by taking only one (1) of my examples
> > does not bolster the value of AWL. In point of fact, I have witnessed
> > AWL -- as you call it, `insanely high AWL adjustment' -- skew mailing
> > list messages, and I began this thread in hopes of understanding how to
> > avoid this and to use AWL to my benefit.
>
> And people have wanted more information about your situation,
> because they haven't experienced anything like the problems you
> have. Thank you for posting the examples. I apologize for
> selecting one of them, but I didn't understand why you were
> posting it or the other four high-scoring examples you posted.
> They were classified as spam, so the system seems to be working
> as intended in those cases (I'm assuming you've already read
> http://wiki.spamassassin.org/w/AwlWrongWay ).
I posted those examples, such as they are, as evidence that I really get
AWL scores that high -- since some skeptics argued that that is not
possible. In fact, so far, everybody has accepted my examples as
examples of high AWL scores. To now argue that they are inadequate
examples of else, is arguably perverse ;>
> Along with those five examples of correctly identified spams,
> you did post two examples where the AWL adjustment may have
> caused a false negative. But in both cases the message
> triggered BAYES_00, which probably had a larger contribution to
> the miscategorization. Judging by those I'd say the real
> problem is spam being incorrectly autolearned as ham. Spam
> should not be getting BAYES_00 or BAYES_01, as it is in the
> majority of your examples. Maybe you should be blaming
> autolearning rather than autowhitelisting.
Hopefully, you notice that _those_ examples also include:
score MDS_Subject_Subscribe +15.0
I'll leave the math to those so inclined ;>
Actually, on further review, I thought that I showed an example with
that rule and its score; but, you get the idea. I have several custom
rules for which I have repeatedly increased the score, because AWL was
taking it away from me. Again, perhaps these type of personal spam are
handled better by procmail.
P.S., Whatever I do know about AWL, how can one message have _both_
BAYES_99 *AND* AWL of -5.4 ???
--
Best Regards,
mds
mds resource
877.596.8237
-
Dare to fix things before they break . . .
-
Our capacity for understanding is inversely proportional to how much
we think we know. The more I know, the more I know I don't know . . .
--
Re: AWL vs. mailing lists
Posted by "Keith C. Ivey" <kc...@cpcug.org>.
Michael D Schleif <md...@helices.org> wrote:
> The problem is in the scores that I cannot present -- those that I have
> -R'd and redisposed -- as I said. Those did *NOT* use
> MDS_Remove_Subject; but, were from same mailing lists. They *ARE* spam;
> but, scored as ham.
They also wouldn't have had an adjustment of -25, which is what
people have been surprised by in your description of your
situation. The -25 results from the very high spam score
caused by the 50-point rule. Without it, the adjustment would
be much less -- though of course it will certainly sometimes
cause messages' scores to be moved from spam range to the
nonspam range.
> Nevertheless, skewing my point by taking only one (1) of my examples
> does not bolster the value of AWL. In point of fact, I have witnessed
> AWL -- as you call it, `insanely high AWL adjustment' -- skew mailing
> list messages, and I began this thread in hopes of understanding how to
> avoid this and to use AWL to my benefit.
And people have wanted more information about your situation,
because they haven't experienced anything like the problems you
have. Thank you for posting the examples. I apologize for
selecting one of them, but I didn't understand why you were
posting it or the other four high-scoring examples you posted.
They were classified as spam, so the system seems to be working
as intended in those cases (I'm assuming you've already read
http://wiki.spamassassin.org/w/AwlWrongWay ).
Along with those five examples of correctly identified spams,
you did post two examples where the AWL adjustment may have
caused a false negative. But in both cases the message
triggered BAYES_00, which probably had a larger contribution to
the miscategorization. Judging by those I'd say the real
problem is spam being incorrectly autolearned as ham. Spam
should not be getting BAYES_00 or BAYES_01, as it is in the
majority of your examples. Maybe you should be blaming
autolearning rather than autowhitelisting.
--
Keith C. Ivey <kc...@cpcug.org>
Washington, DC
Re: AWL vs. mailing lists
Posted by "Keith C. Ivey" <kc...@cpcug.org>.
Bob George <ma...@ttlexceeded.com> wrote:
> Not "evidence", perhaps but SOMETHING caused AWL to give that -25
> adjustment... Again, I see +/- 2 max. adjustment most of the time for
> normal addresses.
As do most people, but if you set up rules in such a way that
you had messages with scores of 50 or so, as Michael does, then
you would see AWL adjustments near -25. The higher the score,
the higher the AWL adjustment will be, because it's half the
difference between the sender's average score and the score for
the current message. It still shouldn't be a problem, because
taking 25 points off a score of 50 still leaves it firmly
within the spam range.
--
Keith C. Ivey <kc...@cpcug.org>
Washington, DC
Re: AWL vs. mailing lists
Posted by Michael D Schleif <md...@helices.org>.
* Bob George <ma...@ttlexceeded.com> [2004:02:25:23:37:49-0500] scribed:
> Michael D Schleif wrote:
> >The problem is in the scores that I cannot present -- those that I have
> >-R'd and redisposed -- as I said. Those did *NOT* use
> >MDS_Remove_Subject; but, were from same mailing lists. They *ARE* spam;
> >but, scored as ham.
>
> So... you're saying AWL adjustments are still being made, even after
> disabling AWL (-a)?
I am not sure yet; but, my point in that quoted statement is that AWL
scores can be both negative *AND* positive. Without AWL, some positive
scores will be *LESS* positive than with AWL. I will learn to deal with
this; but, it remains remark-able ;>
> >NOTE: I raised scores of MDS_Remove_Subject, and its ilk, in response to
> >AWL scores that continually dragged these messages from the ranks of
> >spam into the ranks of ham.
> >
> I am wondering how this even happened. AWL is based on the SENDER
> address, which will (typically) be the actual sender -- not list --
> address. So those individual accounts must've had incredibly high
> non-spam scores. Are some perhaps whitelisted manually?
I, too, am puzzled by this? I have not read the AWL code, nor am I
currently of a mind to do so. Clearly, the `From:' of the subject
mailing list messages is the original sender, not the list. And,
empirically, the AWL scores are highly positive.
> >Of course, I did this before I better
> >understood the AWL process. Perhaps, I should never have used AWL
> >
> If you don't WANT to use AWL, that's fine. But I don't think AWL is
> broken to the point of causing anything like the scoring your're
> seeing... at least not without some help. I'm not trying to say YOU
> should use AWL, but I don't think it's correct to say AWL normally
> causes the behavior you're seeing either. And I'm not convinced AWL on a
> mailing list is a problem.
>
> >perhaps, procmail is a better spam fighting tool than spamassassin ;>
> >
> So's the delete key for that matter! But SA sure makes things a lot EASIER.
>
> >Nevertheless, skewing my point by taking only one (1) of my examples
> >does not bolster the value of AWL. In point of fact, I have witnessed
> >AWL -- as you call it, `insanely high AWL adjustment' -- skew mailing
> >list messages, and I began this thread in hopes of understanding how to
> >avoid this and to use AWL to my benefit.
> >
> ARE YOU WHITELISTING ADDRESSES? Doing any other manual
> tweaks/improvements to the rules?
NO I AM NOT WHITELISTING *A*N*Y* ADDRESSES !?!?
Yes, as you know, I have some custom rules -- some of my own device,
some from the wiki:
~/.spam.MDS.rules.cf
~/.spam.custom.rules.cf
> >Unfortunately, this thread has only driven me from AWL, because its
> >implementation appears skewed, especially in mailing list context.
> >
> Whitelist surely doesn't fit in all scenarios. But again, I'm puzzled as
> to how you got to -25 AWL scores.
Yes, I am also puzzled.
> >And, nobody has offered evidence to the contrary.
> >
> Not "evidence", perhaps but SOMETHING caused AWL to give that -25
> adjustment... Again, I see +/- 2 max. adjustment most of the time for
> normal addresses.
I receive 3,000 - 4,000 email messages per day. Perhaps, the problem
lies in the volume? the mix?
> >Nor has anybody
> >offered a way to reconfigure my AWL to provide value to my environment.
> >I can live without AWL, and I am doing so now.
> >
> Wait, I'm confused. I understood you were having problems even WITHOUT
> AWL. Is something still not working, or just "spammy" messages are
> getting through? What rules are you using?
It is not so much a problem _without_ AWL, as it is an adjustment to
living without AWL. AWL impacts scores both + and -. With AWL, I get
far too many false _negatives_, or spam labeled as ham. Without AWL I
am getting some, a different kind of, false _negatives_, or spam labeled
as grey. At this point, I can easier deal with the latter.
Thank you.
--
Best Regards,
mds
mds resource
877.596.8237
-
Dare to fix things before they break . . .
-
Our capacity for understanding is inversely proportional to how much
we think we know. The more I know, the more I know I don't know . . .
--
Re: AWL vs. mailing lists
Posted by Bob George <ma...@ttlexceeded.com>.
Michael D Schleif wrote:
> [...]
>
>The problem is in the scores that I cannot present -- those that I have
>-R'd and redisposed -- as I said. Those did *NOT* use
>MDS_Remove_Subject; but, were from same mailing lists. They *ARE* spam;
>but, scored as ham.
>
>
So... you're saying AWL adjustments are still being made, even after
disabling AWL (-a)?
>NOTE: I raised scores of MDS_Remove_Subject, and its ilk, in response to
>AWL scores that continually dragged these messages from the ranks of
>spam into the ranks of ham.
>
I am wondering how this even happened. AWL is based on the SENDER
address, which will (typically) be the actual sender -- not list --
address. So those individual accounts must've had incredibly high
non-spam scores. Are some perhaps whitelisted manually?
>Of course, I did this before I better
>understood the AWL process. Perhaps, I should never have used AWL
>
If you don't WANT to use AWL, that's fine. But I don't think AWL is
broken to the point of causing anything like the scoring your're
seeing... at least not without some help. I'm not trying to say YOU
should use AWL, but I don't think it's correct to say AWL normally
causes the behavior you're seeing either. And I'm not convinced AWL on a
mailing list is a problem.
>perhaps, procmail is a better spam fighting tool than spamassassin ;>
>
>
So's the delete key for that matter! But SA sure makes things a lot EASIER.
>Nevertheless, skewing my point by taking only one (1) of my examples
>does not bolster the value of AWL. In point of fact, I have witnessed
>AWL -- as you call it, `insanely high AWL adjustment' -- skew mailing
>list messages, and I began this thread in hopes of understanding how to
>avoid this and to use AWL to my benefit.
>
>
ARE YOU WHITELISTING ADDRESSES? Doing any other manual
tweaks/improvements to the rules?
>Unfortunately, this thread has only driven me from AWL, because its
>implementation appears skewed, especially in mailing list context.
>
>
Whitelist surely doesn't fit in all scenarios. But again, I'm puzzled as
to how you got to -25 AWL scores.
>And, nobody has offered evidence to the contrary.
>
Not "evidence", perhaps but SOMETHING caused AWL to give that -25
adjustment... Again, I see +/- 2 max. adjustment most of the time for
normal addresses.
>Nor has anybody
>offered a way to reconfigure my AWL to provide value to my environment.
>I can live without AWL, and I am doing so now.
>
>
Wait, I'm confused. I understood you were having problems even WITHOUT
AWL. Is something still not working, or just "spammy" messages are
getting through? What rules are you using?
- Bob
Re: AWL vs. mailing lists
Posted by Michael D Schleif <md...@helices.org>.
* "Keith C. Ivey" <kc...@cpcug.org> [2004:02:25:22:03:46-0500] scribed:
> Michael D Schleif <md...@helices.org> wrote:
>
> > pts rule name description
> > ---- ---------------------- --------------------------------------------------
> > 50 MDS_Remove_Subject MDS - un-subscribe idiocy
> > -0.9 BAYES_10 BODY: Bayesian spam probability is 10 to 20%
> > [score: 0.1441]
> > -24 AWL AWL: Auto-whitelist adjustment
>
> Okay, so you have an insanely high AWL adjustment because
> you've set up a rule with an insanely high score. Even so, the
> adjustment only brings the score down to 25, which presumably
> is still far above the spam threshold, so what's the problem?
The problem is in the scores that I cannot present -- those that I have
-R'd and redisposed -- as I said. Those did *NOT* use
MDS_Remove_Subject; but, were from same mailing lists. They *ARE* spam;
but, scored as ham.
NOTE: I raised scores of MDS_Remove_Subject, and its ilk, in response to
AWL scores that continually dragged these messages from the ranks of
spam into the ranks of ham. Of course, I did this before I better
understood the AWL process. Perhaps, I should never have used AWL --
perhaps, procmail is a better spam fighting tool than spamassassin ;>
Nevertheless, skewing my point by taking only one (1) of my examples
does not bolster the value of AWL. In point of fact, I have witnessed
AWL -- as you call it, `insanely high AWL adjustment' -- skew mailing
list messages, and I began this thread in hopes of understanding how to
avoid this and to use AWL to my benefit.
Unfortunately, this thread has only driven me from AWL, because its
implementation appears skewed, especially in mailing list context.
And, nobody has offered evidence to the contrary. Nor has anybody
offered a way to reconfigure my AWL to provide value to my environment.
I can live without AWL, and I am doing so now.
Have a great day!
--
Best Regards,
mds
mds resource
877.596.8237
-
Dare to fix things before they break . . .
-
Our capacity for understanding is inversely proportional to how much
we think we know. The more I know, the more I know I don't know . . .
--
Re: AWL vs. mailing lists
Posted by "Keith C. Ivey" <kc...@cpcug.org>.
Michael D Schleif <md...@helices.org> wrote:
> pts rule name description
> ---- ---------------------- --------------------------------------------------
> 50 MDS_Remove_Subject MDS - un-subscribe idiocy
> -0.9 BAYES_10 BODY: Bayesian spam probability is 10 to 20%
> [score: 0.1441]
> -24 AWL AWL: Auto-whitelist adjustment
Okay, so you have an insanely high AWL adjustment because
you've set up a rule with an insanely high score. Even so, the
adjustment only brings the score down to 25, which presumably
is still far above the spam threshold, so what's the problem?
--
Keith C. Ivey <kc...@cpcug.org>
Washington, DC
Re: AWL vs. mailing lists
Posted by Michael D Schleif <md...@helices.org>.
* "Keith C. Ivey" <kc...@cpcug.org> [2004:02:25:20:26:48-0500] scribed:
> Michael D Schleif <md...@helices.org> wrote:
>
> > In other words, when I turn OFF AWL, the spam in question scores plenty
> > high enough and procmail drops it into my is.spam folder. Otherwise,
> > AWL scores of -25 and lower skew *ALL* mailing list messages.
>
> It seems like something's wrong somewhere, possibly a bug in
> your version of SA. How are you getting an AWL adjustment of
> anything close to -25? Even if the unadjusted spam score for
> the message is 30, which is uncommonly high, the historical
> average for the sender would have to be -20 in order for the
> AWL adjustment to be so large. I thought that the average was
> calculated without whitelist scores, so how is your average
> getting to be so low?
Yes, wrong ;<
My best examples are run through `spamassassin -R', then deposited in my
`not.spam' folder for nightly sa-learn'ing into ham. So, they are *not*
available; but, the following may illustrate my particular problem:
pts rule name description
---- ---------------------- --------------------------------------------------
50 MDS_Remove_Subject MDS - un-subscribe idiocy
-0.9 BAYES_10 BODY: Bayesian spam probability is 10 to 20%
[score: 0.1441]
-24 AWL AWL: Auto-whitelist adjustment
pts rule name description
---- ---------------------- --------------------------------------------------
50 MDS_Subject_Out_of_Office MDS - Out of the Office
-1.5 BAYES_01 BODY: Bayesian spam probability is 1 to 10%
[score: 0.0481]
0.1 HTML_MESSAGE BODY: HTML included in message
-22 AWL AWL: Auto-whitelist adjustment
X-Spam-Checker-Version: SpamAssassin 2.63 (2004-01-11) on
bragi.private.network
X-Spam-Level: **
X-Spam-Status: No, hits=2.7 required=5.0 tests=AWL,BAYES_00,
MDS_Subject_Subscribe,RCVD_IN_SORBS autolearn=no version=2.63
pts rule name description
---- ---------------------- --------------------------------------------------
4.1 FROM_OFFERS From address is "at something-offers"
1.9 REMOVE_REMOVAL_1WORD BODY: List removal information
2.2 YOUR_INCOME BODY: Doing something with my income
3.5 MARKETING_PARTNERS BODY: Claims you registered with a partner
0.2 OFFERS_ETC BODY: Stop the offers, coupons, discounts etc!
0.1 EXCUSE_14 BODY: Tells you how to stop further spam
5.4 BAYES_99 BODY: Bayesian spam probability is 99 to 100%
[score: 0.9992]
0.5 REMOVE_PAGE URI: URL of page called "remove"
1.0 URI_OFFERS URI: Message has link to company offers
0.1 BIZ_TLD URI: Contains a URL in the BIZ top-level domain
0.5 FVGT_u_BIZ_SITE URI: FVGT - contains a URL in the BIZ top-level domain
0.1 CLICK_BELOW Asks you to click below
-5.4 AWL AWL: Auto-whitelist adjustment
pts rule name description
---- ---------------------- --------------------------------------------------
50 MDS_Subject_Out_of_Office MDS - Out of the Office
2.1 BAYES_90 BODY: Bayesian spam probability is 90 to 99%
[score: 0.9625]
-17 AWL AWL: Auto-whitelist adjustment
pts rule name description
---- ---------------------- --------------------------------------------------
50 MDS_Subject_Out_of_Office MDS - Out of the Office
-4.9 BAYES_00 BODY: Bayesian spam probability is 0 to 1%
[score: 0.0000]
-20 AWL AWL: Auto-whitelist adjustment
X-Spam-Checker-Version: SpamAssassin 2.61 (1.212.2.1-2003-12-09-exp) on
bragi.private.network
X-Spam-Level: **
X-Spam-Status: No, hits=2.6 required=5.0 tests=AWL,BAYES_00,
MDS_Subject_Subscribe autolearn=no version=2.61
--
Best Regards,
mds
mds resource
877.596.8237
-
Dare to fix things before they break . . .
-
Our capacity for understanding is inversely proportional to how much
we think we know. The more I know, the more I know I don't know . . .
--
Re: AWL vs. mailing lists
Posted by "Keith C. Ivey" <kc...@cpcug.org>.
Michael D Schleif <md...@helices.org> wrote:
> In other words, when I turn OFF AWL, the spam in question scores plenty
> high enough and procmail drops it into my is.spam folder. Otherwise,
> AWL scores of -25 and lower skew *ALL* mailing list messages.
It seems like something's wrong somewhere, possibly a bug in
your version of SA. How are you getting an AWL adjustment of
anything close to -25? Even if the unadjusted spam score for
the message is 30, which is uncommonly high, the historical
average for the sender would have to be -20 in order for the
AWL adjustment to be so large. I thought that the average was
calculated without whitelist scores, so how is your average
getting to be so low?
--
Keith C. Ivey <kc...@cpcug.org>
Washington, DC
Re: AWL vs. mailing lists
Posted by Bob George <ma...@ttlexceeded.com>.
Michael D Schleif wrote:
>* Bob George <ma...@ttlexceeded.com> [2004:02:25:21:20:55-0700] scribed:
>
>
> [...]
>
>>You mean that most messages are now somewhat positive (spam)? What
>>do you mean by positive counterparts? Is something being ADDED to
>>the scores, or just no AWL negative adjustment? (Sorry, just
>>confused here.)
>>
>>
>
>I mean that AWL generates *both* positive _and_ negative scores.
>
>The negative scores were perplexing me. I have turned OFF AWL, and now
>I realize the affects of the positive AWL scores. I have a grey zone
>between scores of 5 and 6, where I am un-certain about their spam-ness.
>With AWL, I had very few (2-3/day) grey zone messages. Without AWL, it
>looks like I may see 3-5x that number ;<
>
>
Ah, so formerly "spam" messages are now scored borderline and thus under
the threshold?
Are you using any of the add-on rule sets? Bayes? Until bayes kicks in,
there are many messages that I see that might squeak past the defaults.
- Bob
Re: AWL vs. mailing lists
Posted by Michael D Schleif <md...@helices.org>.
* Bob George <ma...@ttlexceeded.com> [2004:02:25:21:20:55-0700] scribed:
> "Michael D Schleif" <md...@helices.org> wrote:
<snip />
> > I have turned OFF AWL completely, and have already noticed that AWL
> > negative scores have positive counterparts. Now, lack of those
> > positive AWL scores are increasing my grey-area scores (5 - 6).
>
> You mean that most messages are now somewhat positive (spam)? What
> do you mean by positive counterparts? Is something being ADDED to
> the scores, or just no AWL negative adjustment? (Sorry, just
> confused here.)
I mean that AWL generates *both* positive _and_ negative scores.
The negative scores were perplexing me. I have turned OFF AWL, and now
I realize the affects of the positive AWL scores. I have a grey zone
between scores of 5 and 6, where I am un-certain about their spam-ness.
With AWL, I had very few (2-3/day) grey zone messages. Without AWL, it
looks like I may see 3-5x that number ;<
> > What do you think?
>
> I think that -25 AWL adjustment is at the heart of the problem!
See my other post.
--
Best Regards,
mds
mds resource
877.596.8237
-
Dare to fix things before they break . . .
-
Our capacity for understanding is inversely proportional to how much
we think we know. The more I know, the more I know I don't know . . .
--
Re: AWL vs. mailing lists
Posted by Bob George <ma...@ttlexceeded.com>.
"Michael D Schleif" <md...@helices.org> wrote:
> Yesterday, I received 414 messages from debian mailing lists.
By far,
> the most voluminous is debian-user. Plus, debian lists are
targets for
> spammers -- I do not understand the rationale; but, I have quite
the
> corpus from those lists. Worse, I have other mailing list
examples.
I was once subscribed to that list, but found it a bit
overwhelming. :)
Aren't messages still posted with From: set to the original
sender? Or are they From: a moderator address (making me wonder
how spam got out)?
> Yes, I know. All those (apparently) do are zero out the current
AWL,
> then build again.
I'm leaning more and more towards different 'policy' for lists.
AWL works for me, but thinking about it, I can see where I might
not want to 'trust' it for more public forums. I'm considering
either calling spamassassin (rather than spamc) without the -a
option, or running another spamd without it (which I need to
test).
Running messages through both in parallel might be interesting.
> I have turned OFF AWL completely, and have already noticed that
AWL
> negative scores have positive counterparts. Now, lack of those
positive
> AWL scores are increasing my grey-area scores (5 - 6).
You mean that most messages are now somewhat positive (spam)? What
do you mean by positive counterparts? Is something being ADDED to
the scores, or just no AWL negative adjustment? (Sorry, just
confused here.)
> What do you think?
I think that -25 AWL adjustment is at the heart of the problem!
BUT, if AWL isn't gaining you anything, there's no need to run it
either.
- Bob
--
Best Regards,
mds
mds resource
877.596.8237
-
Dare to fix things before they break . . .
-
Our capacity for understanding is inversely proportional to how
much
we think we know. The more I know, the more I know I don't know .
. .
--
Re: AWL vs. mailing lists
Posted by Michael D Schleif <md...@helices.org>.
* Bob George <ma...@ttlexceeded.com> [2004:02:25:20:27:00-0700] scribed:
> "Michael D Schleif" <md...@helices.org> wrote:
> > OK, I am probably going to turn OFF AWL for my environment, because
> > it is becoming clear to me that the value added is not enough to
> > warrant it.
>
> Is this a personal, or multi-user system you're running? I'm
> curious how a mailing list address wound up with SUCH a negative
> score, unless you specifically whitelisted it.
Yesterday, I received 414 messages from debian mailing lists. By far,
the most voluminous is debian-user. Plus, debian lists are targets for
spammers -- I do not understand the rationale; but, I have quite the
corpus from those lists. Worse, I have other mailing list examples.
> > When I do this, I will no longer have _any_ AWL scores in
> > subsequent messages? I do not need to do `spamassassin -R' on a
> > corpus of mail?
>
> I did some quick testing after your previous message, and AWL
> scores STILL showed after doing
> both --remove-addr-from-whitelist=list@ddress and -R on list
> messages, calling spamassassin without -a. I'm not sure how to GET
> RID of existing AWL scores! I'd be interested in hearing from
> anyone who has myself.
Yes, I know. All those (apparently) do are zero out the current AWL,
then build again.
I have turned OFF AWL completely, and have already noticed that AWL
negative scores have positive counterparts. Now, lack of those positive
AWL scores are increasing my grey-area scores (5 - 6).
What do you think?
--
Best Regards,
mds
mds resource
877.596.8237
-
Dare to fix things before they break . . .
-
Our capacity for understanding is inversely proportional to how much
we think we know. The more I know, the more I know I don't know . . .
--
Re: AWL vs. mailing lists
Posted by Bob George <ma...@ttlexceeded.com>.
"Michael D Schleif" <md...@helices.org> wrote:
> OK, I am probably going to turn OFF AWL for my environment,
because it
> is becoming clear to me that the value added is not enough to
warrant
> it.
Is this a personal, or multi-user system you're running? I'm
curious how a mailing list address wound up with SUCH a negative
score, unless you specifically whitelisted it.
> When I do this, I will no longer have _any_ AWL scores in
> subsequent messages? I do not need to do `spamassassin -R' on a
corpus
> of mail?
I did some quick testing after your previous message, and AWL
scores STILL showed after doing
both --remove-addr-from-whitelist=list@ddress and -R on list
messages, calling spamassassin without -a. I'm not sure how to GET
RID of existing AWL scores! I'd be interested in hearing from
anyone who has myself.
- Bob
Re: AWL vs. mailing lists
Posted by Michael D Schleif <md...@helices.org>.
* Gary Funck <ga...@intrepid.com> [2004:02:25:15:27:11-0800] scribed:
>
> > From: Michael D. Schleif [mailto:mds@bragi.private.network]On Behalf Of
> > Michael D Schleif
> > Sent: Wednesday, February 25, 2004 11:58 AM
> >
> > In other words, when I turn OFF AWL, the spam in question scores plenty
> > high enough and procmail drops it into my is.spam folder. Otherwise,
> > AWL scores of -25 and lower skew *ALL* mailing list messages.
> >
> > What am I missing?
>
> Think about AWL a moment: in the limit, it let's people you trust send
> you an occasional spam. Is that what you want? I've got a friend, who
> swears like a sailor, and tosses me a risque' message every now and
> then, that used to bounce off the walls back in the days that I used a
> home-grown spam-phrase checking script to fend off spam. Because of
> him, I had to introduce a white list. And in the new, SA, scheme of
> things, I think there's still a place for explicit whitelisting. And,
> especially when Bayes is trained and tuned, I don't see much of a case
> for AWL, at all. Bayes will notice which from addresses and other
> attributes typically indicate ham, so why add another level of
> checking that sometimes works against the other mechanisms in place?
OK, I am probably going to turn OFF AWL for my environment, because it
is becoming clear to me that the value added is not enough to warrant
it. When I do this, I will no longer have _any_ AWL scores in
subsequent messages? I do not need to do `spamassassin -R' on a corpus
of mail?
> BTW, you could tune up your procmail script to not call spamassassin
> with AWL enabled, when receiving e-mail from mailing lists.
>
> (refer to: http://www.professional.org/procmail/listname_id.rc)
>
> AWL="-a"
>
> INCLUDERC="/etc/procmailrcs/listname_id.rc"
>
> :0
> * ! LISTNAME ?? ^^^^
> { AWL = "" }
>
> :0 fw:spam.lock
> | spamassassin "$AWL"
This looks interesting, and I had started to piece this together.
Nevertheless, it looks like `automatic' white listing is not a good fit
with my overall strategy.
Thank you.
--
Best Regards,
mds
mds resource
877.596.8237
-
Dare to fix things before they break . . .
-
Our capacity for understanding is inversely proportional to how much
we think we know. The more I know, the more I know I don't know . . .
--
Re: AWL vs. mailing lists
Posted by Bob George <ma...@ttlexceeded.com>.
"Michael D Schleif" <md...@helices.org> wrote:
> Nevertheless, I have an elaborate spam/virus plan, just as you and Gary
> do.
Ah, not trying to preach. Just tossing out ideas for discussion.
> I think that you missed the intent of my post. Like Gary, I am becoming
> disillusioned with AWL results. I am wondering if there is something I
> can do to _enhance_ AWL, or should I just turn it OFF?
Ah, OK. I got thrown by the "virus got by" aspect of the discussion...
> In the non-mailing list context, I currently have no objections with
> AWL. The objections to AWL that I have appear only in mailing list
> context.
So you'd ideally like SELECTIVE disabling of AWL?
I'm playing with --remove-addr-from-whitelist=addr but I'm not sure if the
effect is permanent or not. I ran a list message through spamassassin -R, and
re-ran checks. It claims the address is removed, yet an AWL adjustment still
shows when scoring a message.
> If I understand how spamassassin currently implements AWL,
> this all makes sense: mailing lists, as senders of email, tend to have
> extremely high ham incidence, and therefore, when they do emit spam, it
> is extremely skewed by AWL.
Hmm... I was thinking a meta-rule for list-headers and AWL might work, but a
quick test makes me think not.
A procmail rule that checks for list headers (i.e. List-Id:) then calls
spamassassin without -a perhaps? (Back to the use of procmail for policy-based
decisions).
> In other words, when I turn OFF AWL, the spam in question scores plenty
> high enough and procmail drops it into my is.spam folder. Otherwise,
> AWL scores of -25 and lower skew *ALL* mailing list messages.
After some quick checking, it seems most of my list AWL adjustments are small
(.1 or so). To be a factor, the adjustment would have to be significant,
implying the normal list is HIGHLY non-spammy, right? Is that list adderss
perhaps whitelisted? -25 seems very high to me. So the message is scoring +25
or more without AWL right?
- Bob
RE: AWL vs. mailing lists
Posted by "Keith C. Ivey" <kc...@cpcug.org>.
Gary Funck <ga...@intrepid.com> wrote:
> Although I agree with your overall description of AWL, it doesn't in fact
> let my friend's messages through because his messages often score higher.
> If the often score high, his AWL score will be high (more spammy), and as
> it creeps up will actually increase the chances that his message goes above
> the spam threshold.
I think you're misunderstanding how AWL works. There's no way
it will put your friend's message over the spam threshold
unless the average score for your friend's previous messages is
above the spam threshold (and probably significantly above,
unless the new message is already close to the threshold). If
you have a friend that consistently sends you messages that
score above the spam threshold, then you need to manually
whitelist that friend.
--
Keith C. Ivey <kc...@cpcug.org>
Washington, DC
Re: AWL vs. mailing lists
Posted by Bob George <ma...@ttlexceeded.com>.
Rich Puhek <rp...@etnsystems.com> wrote:
> Good call. Although that's a bit of a pain site wide.
> Wondering how a program like SA could automatically determine
> that joe@sailor.org is legit. A semi-automatic way would be if
> the Bayes learn process also updated the AWL... ham senders
> get a AWL ham bias spam senders get an AWL spam bias.
Hmm. I have bayes auto turned on, and I don't think it does that. However, I
also capture all flagged spam for bayes training (other bayes tools), so
extracting the offending sender and doing --add-to-blacklist= works, though
it's probably futile given that they rotate adresses. Maybe just REMOVE those
addresses to make sure they don't clutter up the AWL.
In the case of "good" posters, one could easily set up an "add to whitelist" in
the same way a bayes training folder is done. The folder gets scanned
periodically, and the sender extracted and fed to spamassassin
with --add-to-whitelist=.
I see "add-to-whitelist", "remove-from-whitelist", "add-to-blacklist" and
"remove-from-blacklist" folders. Cumbersome, but any of the techniques for
bayes updating could be adapted.
> Don't
> know how practical that is versus just manually maintaining
> whitelist entries.
Scalability seems to be the problem. Adapting a proven bayes-training approach
is probably easiest.
- Bob
RE: AWL vs. mailing lists
Posted by Gary Funck <ga...@intrepid.com>.
> From: Rich Puhek
> Sent: Thursday, February 26, 2004 7:57 AM
[...]
> > My approach to this is to outright whielist joe@sailor.org.
> Bayes ignores
> > whitelisting as I recall, in deciding whether to autolearn or not.
> >
> > Same would go for broker@mortgage.com - I just add him to my explicit
> > white list.
> >
> Good call. Although that's a bit of a pain site wide. Wondering how a
> program like SA could automatically determine that joe@sailor.org is
> legit. A semi-automatic way would be if the Bayes learn process also
> updated the AWL... ham senders get a AWL ham bias spam senders get an
> AWL spam bias. Don't know how practical that is versus just manually
> maintaining whitelist entries.
>
I agree. Manual whitelisting is a pain, and so user-level re-categorization
of misfile ham/spam. What is needed is a nice GUI for managing those tasks.
Regarding whitelisting, it's been mentioned before (and I agree that it
sounds
promising) to look at the outgoing mail stream for whitelisting clues. If we
assume that the users generally only send mail to those addresses they wish
to engage in a conversation with, then one should bias incoming mail as
likely
ham, if it is from an address that was originally mentioned in an outgoing
mail.
This feedback loop could be accomplished either as an MTA filter, or
possibly
just a program that monitors the mail logs.
A simple Bayes system that learns spam only from spam traps, and learns ham
only from the outgoing mail stream (some adjustments would have to be made
for things like from/to addresses and Received lines) might be an
interesting
cut at a totally automated set up. That works until one of the users decides
to
go into the spam mailing business, or his PC is co-opted by a spammer's
worm.
Re: AWL vs. mailing lists
Posted by Rich Puhek <rp...@etnsystems.com>.
Gary Funck wrote:
>
> Although I agree with your overall description of AWL, it doesn't in fact
> let my friend's messages through because his messages often score higher.
> If the often score high, his AWL score will be high (more spammy), and as
> it creeps up will actually increase the chances that his message goes above
> the spam threshold.
>
Got it... hmmm, I suppose that's because he sends more "spammy looking"
mail than non-spammy. If he normally sent very clearly hammy messages,
then sends a very blue joke email once in a while, AWL would probably
work as intended.
>
>>The idea does break a bit with mailing lists and with AWL poisoning
>>attempts, of course. For that, your procmail example looks perfect.
>>
>
>
> The idea partially breaks with mailing lists, because often the spammers
> post their very first message to the list and it is spam. Same would
> go for mary@blandagency.gov, if the first message she sent you was spam.
> Maybe AWL should start off at 1/5 of the spam threshold, thus 1.0 on a 5.0
> scale, slightly biasing in favor of spam upon first arrivals?
>
Or at least do a slow start, where AWL doesn't really kick in until it
sees a few messages.
> I don't think it happens that often that someone who has posted to a mailing
> list over a long period of time suddenly turns around and sends a spam
> (though it does happen, and some spammers are even using this trick). What
> makes mailing lists different is that there are many more senders, and on
> overage you don't see very many messages from a given sender. Again, maybe
> AWL should begin with a bias in favor of judging initial messages as spam?
>
>
>>The other nice thing about the AWL approach vs relying on Bayes...
>>suppose your friend sends you enough risqué messages that traditional
>>porn spam indicators are no longer reliable spam tokens?
>
>
> My approach to this is to outright whielist joe@sailor.org. Bayes ignores
> whitelisting as I recall, in deciding whether to autolearn or not.
>
> Same would go for broker@mortgage.com - I just add him to my explicit
> white list.
>
Good call. Although that's a bit of a pain site wide. Wondering how a
program like SA could automatically determine that joe@sailor.org is
legit. A semi-automatic way would be if the Bayes learn process also
updated the AWL... ham senders get a AWL ham bias spam senders get an
AWL spam bias. Don't know how practical that is versus just manually
maintaining whitelist entries.
--Rich
Re: AWL vs. mailing lists
Posted by Bob George <ma...@ttlexceeded.com>.
Gary Funck wrote:
>> From: Rich Puhek [mailto:rpuhek@etnsystems.com]
> [...]
>> Seems like AWL is indeed intended to let people I trust
occasionally
>> send me email that _SA thinks is spam_ (slight difference from
your
>> question).
And vice-versa I suppose.
>> So the idea is that the AWL system figures out that your
>> sailor buddy's messages tend to score high, and AWL compensates
for
>> it automatically (no training needed).
>
> Although I agree with your overall description of AWL, it
doesn't in
> fact let my friend's messages through because his messages often
> score higher. If the often score high, his AWL score will be
high
> (more spammy), and as it creeps up will actually increase the
chances
> that his message goes above the spam threshold.
I think a manual whitelist entry is probably more appropriate for
those spammy friends... and this certainly raises questions about
the wisdom of auto-bayes training off ALL ham.
But yes, my understanding is that it brings messages more towards
the overall norm for the sender in terms of scoring.
>> The idea does break a bit with mailing lists and with AWL
poisoning
>> attempts, of course. For that, your procmail example looks
perfect.
>
> The idea partially breaks with mailing lists, because often the
> spammers post their very first message to the list and it is
spam.
Hmm. In the case of a one-off poster, does AWL come into play?
Hopefully, any frequent spammer will be moderated or banned from a
list. It varies by list, but most I'm on have the From: field set
to the original sender, and THIS is the address considered in AWL.
So messages to the list FROM FREQUENT POSTERS are affected by AWL,
not all.
I've yet to see an AWL adjustment of more than -2, and certainly
nothing like the -25 described by Michael
> Same would
> go for mary@blandagency.gov, if the first message she sent you
was
> spam. Maybe AWL should start off at 1/5 of the spam threshold,
thus
> 1.0 on a 5.0 scale, slightly biasing in favor of spam upon first
> arrivals?
Yes (based on what I know -- admittedly only as a lowly end-user).
> I don't think it happens that often that someone who has posted
to a
> mailing list over a long period of time suddenly turns around
and
> sends a spam (though it does happen, and some spammers are even
using
> this trick).
I suppose forged messages would be a problem. Still, other SA
methods would, I think, generally over-ride a +/-2 offset.
> What makes mailing lists different is that there are
> many more senders, and on overage you don't see very many
messages
> from a given sender.
So AWL really won't come into play will it? On the busiest lists
I'm on, AWL adjustments are generally -1 - 0 range. Minor
"nudges". I'd only really expect AWL to matter if a VERY trusted,
frequent sender sends (or forwards) the odd spam.
> Again, maybe AWL should begin with a bias in
> favor of judging initial messages as spam?
It starts with none at all, which seems to me, prudent.
- Bob
>> The other nice thing about the AWL approach vs relying on
Bayes...
>> suppose your friend sends you enough risqué messages that
traditional
>> porn spam indicators are no longer reliable spam tokens?
>
> My approach to this is to outright whielist joe@sailor.org.
Bayes
> ignores whitelisting as I recall, in deciding whether to
autolearn or
> not.
>
> Same would go for broker@mortgage.com - I just add him to my
explicit
> white list.
RE: AWL vs. mailing lists
Posted by Gary Funck <ga...@intrepid.com>.
> From: Rich Puhek [mailto:rpuhek@etnsystems.com]
> Sent: Wednesday, February 25, 2004 4:26 PM
[...]
>
> But Bayes doesn't know that "bob@sailor.org" tends to send messages that
> score 10 points while "mary@blandagency.gov" tends to send messages that
> score 0 points. It may be able to tell that "bob@sailor.org" is a ham
> token, assuming that you've trained Bob's messages as ham.
>
> Seems like AWL is indeed intended to let people I trust occasionally
> send me email that _SA thinks is spam_ (slight difference from your
> question). So the idea is that the AWL system figures out that your
> sailor buddy's messages tend to score high, and AWL compensates for it
> automatically (no training needed).
>
Although I agree with your overall description of AWL, it doesn't in fact
let my friend's messages through because his messages often score higher.
If the often score high, his AWL score will be high (more spammy), and as
it creeps up will actually increase the chances that his message goes above
the spam threshold.
> The idea does break a bit with mailing lists and with AWL poisoning
> attempts, of course. For that, your procmail example looks perfect.
>
The idea partially breaks with mailing lists, because often the spammers
post their very first message to the list and it is spam. Same would
go for mary@blandagency.gov, if the first message she sent you was spam.
Maybe AWL should start off at 1/5 of the spam threshold, thus 1.0 on a 5.0
scale, slightly biasing in favor of spam upon first arrivals?
I don't think it happens that often that someone who has posted to a mailing
list over a long period of time suddenly turns around and sends a spam
(though it does happen, and some spammers are even using this trick). What
makes mailing lists different is that there are many more senders, and on
overage you don't see very many messages from a given sender. Again, maybe
AWL should begin with a bias in favor of judging initial messages as spam?
> The other nice thing about the AWL approach vs relying on Bayes...
> suppose your friend sends you enough risqué messages that traditional
> porn spam indicators are no longer reliable spam tokens?
My approach to this is to outright whielist joe@sailor.org. Bayes ignores
whitelisting as I recall, in deciding whether to autolearn or not.
Same would go for broker@mortgage.com - I just add him to my explicit
white list.
Re: AWL vs. mailing lists
Posted by Rich Puhek <rp...@etnsystems.com>.
Gary Funck wrote:
> Think about AWL a moment: in the limit, it let's people you trust send
> you an occasional spam. Is that what you want? I've got a friend, who swears
> like a sailor, and tosses me a risque' message every now and then, that
> used to bounce off the walls back in the days that I used a home-grown
> spam-phrase checking script to fend off spam. Because of him, I had to
> introduce
> a white list. And in the new, SA, scheme of things, I think there's still a
> place
> for explicit whitelisting. And, especially when Bayes is trained and tuned,
> I don't
> see much of a case for AWL, at all. Bayes will notice which from addresses
> and
> other attributes typically indicate ham, so why add another level of
> checking that
> sometimes works against the other mechanisms in place?
But Bayes doesn't know that "bob@sailor.org" tends to send messages that
score 10 points while "mary@blandagency.gov" tends to send messages that
score 0 points. It may be able to tell that "bob@sailor.org" is a ham
token, assuming that you've trained Bob's messages as ham.
Seems like AWL is indeed intended to let people I trust occasionally
send me email that _SA thinks is spam_ (slight difference from your
question). So the idea is that the AWL system figures out that your
sailor buddy's messages tend to score high, and AWL compensates for it
automatically (no training needed).
The idea does break a bit with mailing lists and with AWL poisoning
attempts, of course. For that, your procmail example looks perfect.
The other nice thing about the AWL approach vs relying on Bayes...
suppose your friend sends you enough risqué messages that traditional
porn spam indicators are no longer reliable spam tokens? With AWL, at
least there's an association with a specific sender, so your friend's
emails get through, but porn spam gets stopped. The same arguments can
be applied to stocks, prescription medications, mortgages, and even just
HTML email from friends on hotmail.
If the Bayes system was able to associate per sender (so that "new low
rates" was ok from my mortgage broker, but not from just anyone) Bayes
might supersede AWL, but that doesn't seem practical to me.
--Rich
RE: AWL vs. mailing lists
Posted by Gary Funck <ga...@intrepid.com>.
> From: Michael D. Schleif [mailto:mds@bragi.private.network]On Behalf Of
> Michael D Schleif
> Sent: Wednesday, February 25, 2004 11:58 AM
>
> In other words, when I turn OFF AWL, the spam in question scores plenty
> high enough and procmail drops it into my is.spam folder. Otherwise,
> AWL scores of -25 and lower skew *ALL* mailing list messages.
>
> What am I missing?
Think about AWL a moment: in the limit, it let's people you trust send
you an occasional spam. Is that what you want? I've got a friend, who swears
like a sailor, and tosses me a risque' message every now and then, that
used to bounce off the walls back in the days that I used a home-grown
spam-phrase checking script to fend off spam. Because of him, I had to
introduce
a white list. And in the new, SA, scheme of things, I think there's still a
place
for explicit whitelisting. And, especially when Bayes is trained and tuned,
I don't
see much of a case for AWL, at all. Bayes will notice which from addresses
and
other attributes typically indicate ham, so why add another level of
checking that
sometimes works against the other mechanisms in place?
BTW, you could tune up your procmail script to not call spamassassin with
AWL enabled,
when receiving e-mail from mailing lists.
(refer to: http://www.professional.org/procmail/listname_id.rc)
AWL="-a"
INCLUDERC="/etc/procmailrcs/listname_id.rc"
:0
* ! LISTNAME ?? ^^^^
{ AWL = "" }
:0 fw:spam.lock
| spamassassin "$AWL"
Re: AWL vs. mailing lists
Posted by Michael D Schleif <md...@helices.org>.
* Bob George <ma...@ttlexceeded.com> [2004:02:25:14:01:30-0500] scribed:
> "Michael D Schleif" <md...@helices.org> wrote:
>
> > I subscribe to over one hundred (100) mailing lists. procmail processes
> > every incoming message <256kB through SA, unless procmail dumps it
> > first.
>
> In the same boat here, list-wise.
>
> > High volume mailing lists (e.g., debian-user) have an extremely high
> > incidence of ham; therefore, AWL is extremely high for such lists. When
> > the virus/worm-of-the-day spams the list, SA identifies these as spam,
> > then transforms them into ham -- solely due to AWL skewing ;<
>
> Keep in mind that relying on SA or bayes as your SOLE means of filtering
> viruses and worms can be risky. They can be USEFUL certainly, but nasty
> surprises are going to come in a variety of "non-spammish" packaging. And SA IS
> only checking messages smaller than 256KB, so it's not a complete solution in
> any case.
<snip />
Thank you, for your insightful comments.
Nevertheless, I have an elaborate spam/virus plan, just as you and Gary
do.
I think that you missed the intent of my post. Like Gary, I am becoming
disillusioned with AWL results. I am wondering if there is something I
can do to _enhance_ AWL, or should I just turn it OFF?
In the non-mailing list context, I currently have no objections with
AWL. The objections to AWL that I have appear only in mailing list
context. If I understand how spamassassin currently implements AWL,
this all makes sense: mailing lists, as senders of email, tend to have
extremely high ham incidence, and therefore, when they do emit spam, it
is extremely skewed by AWL.
In other words, when I turn OFF AWL, the spam in question scores plenty
high enough and procmail drops it into my is.spam folder. Otherwise,
AWL scores of -25 and lower skew *ALL* mailing list messages.
What am I missing?
--
Best Regards,
mds
mds resource
877.596.8237
-
Dare to fix things before they break . . .
-
Our capacity for understanding is inversely proportional to how much
we think we know. The more I know, the more I know I don't know . . .
--
Re: AWL vs. mailing lists
Posted by Bob George <ma...@ttlexceeded.com>.
"Michael D Schleif" <md...@helices.org> wrote:
> I subscribe to over one hundred (100) mailing lists. procmail processes
> every incoming message <256kB through SA, unless procmail dumps it
> first.
In the same boat here, list-wise.
> High volume mailing lists (e.g., debian-user) have an extremely high
> incidence of ham; therefore, AWL is extremely high for such lists. When
> the virus/worm-of-the-day spams the list, SA identifies these as spam,
> then transforms them into ham -- solely due to AWL skewing ;<
Keep in mind that relying on SA or bayes as your SOLE means of filtering
viruses and worms can be risky. They can be USEFUL certainly, but nasty
surprises are going to come in a variety of "non-spammish" packaging. And SA IS
only checking messages smaller than 256KB, so it's not a complete solution in
any case.
I would say that the problem is that a worm/virus got into your mail queue
rather than spamassassin scoring it wrong. Based on my recent experiences using
SA, various bayes-tools, procmail and anti-virus (clamav, bitdefender), I'm
increasingly convinced that MULTIPLE layers of defense are the best bet:
1. Coarse, fast filtering with procmail (enforce policy - see below)
2. Scanning of attachments/messages for virus/worms using a tool with modern
capabilities for the job (heuristics)
3. Detection of spam in otherwise "clean" mail with SA to keep mail tidy
4. Seletive defanging/quarantine (i.e. anomy sanitizer) (enforce policy - see
below)
I'm leaning towards a "policy-based" approach. If attachments (or html content)
don't belong on a list/to a user, filter/defang it. This actually works well
for my mailing lists, as I can simply put a defang/quarantine filter in front
of them. While it's nice that SA may detect something, an executable doesn't
belong on (most of) those lists, so I don't want to let it through unless I
create a specific exception. Depending on how testing goes, it might be
simplified to "no attachments on messages (not directly addressed to
individuals with local accounts|with headers indicating a mailing list)" or
some variation.
Policy may allow some attachments (i.e. pictures) but not others (i.e. MP3s)
for some users/to some lists.
Each tool has its place, and all are complimentary:
* Procmail is great for doing fast filtering of messages
* SpamAssassin is great at detecting existing and emerging spam patterns, based
on a user's specific preferences.
* Anti-Virus tools are great at detecting existing and emerging virus/worms,
based on heuristics and other techniques.
* Defang/quarantine is useful for discriminating among types of content.
Just my thoughts.
- Bob
Re: AWL vs. mailing lists
Posted by Bob George <ma...@ttlexceeded.com>.
Michael,
I think I know what happened with spams getting through with high AWL scores to
some lists after reading some recent messages on related topics.
Check one of those spams, particularly the received headers. See if any in the
path that message took are listed in /usr/share/spamassassin/60_whitelist.cf.
If so, messages taking that path are whitelisted, meaning they start with
a -100 score. When the message hits AWL, THAT SENDER (the spammer) starts
with -100, regardless of the score, so their average IS highly-anti-spam. Their
next message (which there probably won't be one) will be scored more spammy,
but still not as spam. It's only after several posts that the AWL adjustment
will be corrected.
If this is correct, that same spammer could probably get a message through
based on DEFAULT WHITELIST settings, regardless of AWL. Have you seen repeats
of this problem?
I'm increasingly convinced that the default whitelist is risky for this reason.
- Bob
Re: More testing and a possible fix for skewed AWL scoring (was Re:
AWL vs. mailing lists)
Posted by Bob George <ma...@ttlexceeded.com>.
Advance warning: I am NOT a spamassassin expert, just a determined
end-user. Please correct any mistakes I make in what follows!
Michael D Schleif wrote:
> [...]
>
>This exposition makes more sense to me than anything else in this
>thread! Thank you. I suppose, on further reflection, other data
>posited will make its way into my brain as lucid information, too ;>
>
>
I'm glad it's useful. Exploring this has given me a much better
appreication for the inner workings of SA, and it's still among my
favorite "discoveries" among open-source solutions. (I feel like
Columbus saying that.)
>[...] Once per night, I process like this:
>
> # cat ~/mail/errata/teach.spam.sh
> #!/bin/sh
> # *NOT* spam
> sa-learn --no-rebuild --ham --mbox ~/mail/errata/.spam/not.spam
> # *IS* spam
> sa-learn --no-rebuild --spam --mbox ~/mail/errata/.spam/is.spam
> # Re-Build database with latest expertise
> sa-learn --rebuild --force-expire
> exit 0
>
>
I do something similar, except I train other bayes tools (bogofilter,
etc.) in same script. I *do* have bayes-autolearn enabled (default
settings). The only mis-classified messages I see regularly are well
within those ranges, so this hasn't been a problem. SA bayes is scoring
VERY accurately for me.
>In the cases where I have attributed *WRONG* classification of spam to
>AWL, I also do this:
>
> |spamassassin -R
>
>
I believe you're really after --remove-addr-from-whitelist=. -R removes
ALL addresses noted in the message. I don't think it's parsing a mail
message so much as just scanning for @ address patterns.
>Please, comment on the efficacy of this process.
>
>Much discussion on this list about AWL asserts that _only_ the sender
>(From:) of the message is subject to AWL; but, I find this:
>
>
I believe that is correct, based on my testing using check_whitelist.
>SpamAssassin auto-whitelist: removing address: tme10@lists.us.ibm.com
>[...]
>These are the implicated headers:
>
> Return-path: owner-tme10@lists.us.ibm.com
> From: ***** ***** <**...@us.ibm.com>
> To: tme10@lists.us.ibm.com
> Message-ID: <OF...@us.ibm.com>
> Sender: owner-tme10@lists.us.ibm.com
>
>I do not know whether it gets owner-tme10@lists.us.ibm.com from
>Return-path: or Sender: -- which is it? And, how does it get my
>address: mds@helices.org -- the headers only us it in Received: headers?
>
>
-R will take out addresses in ANY portion of the message, per the manpage.
>So, clearly, AWL score is a function of many disparate data, and I
>persist in my belief that voluminous mailing lists are poor applications
>for AWL.
>
>
I think you're misinterpreting the usage of -R. Per manpage:
-R, --remove-from-whitelist
Remove all email addresses, in the headers and body of the mail
message read from STDIN, from the automatic whitelist.
Otherwise, you'd see every address in any of the aforementioned headers
(plus body etc.) if -R were truly indicative of what's added via AWL) in
your whitelist, no?
>Prior to Bob's message, I could not find check_whitelist. Tonight, I
>took the time, found it and used it. Please, notice that what follows
>is post ad hoc spamassassin -R'ing and willy nilly adjustment of my
>rules. However, let it be known that I have *NOT* adjusted black/white
>lists, with the sole exception of the horrible legacy score for
>amazon.com.
>
>
The problem for me at least was that ANY prior use of black/whitelist
resulted in adjusted AWL scores, with effects lingering even after the
actual black/whitelist entry is removed. Note that any "insanely scored"
rule would have similar effects! (Moral: use LOW rule scores to nudge,
not hammer.)
The true test of AWL is how posts from users score AFTER the existence
of black/whitelist entries (in ANY .cf parsed) and AFTER the addresses
have been removed from the autowhitelist file of THE USER RUNNING EITHER
SPAMASSASSIN or SPAMD.
>Please, somebody help me understand these:
>
># /usr/share/doc/spamassassin/examples/check_whitelist | sort -nr | head -n 50 | sed 's![^ ]*@!*****@!'
> 662.2 (662.2/3) -- *****@hp.com|ip=15.1
>
>
That's 662.2 average for THREE posted messages. If this is a repeat
spammer address (verify), then that might be perfectly reasonable (they
only send mega-high-scoring crap). In my case, the only such entry
anywhere near that level is from a test account I use for testing
spamassassin rules... though I am curious how it got THAT high!
401.1 (401.1/5) -- *****@iname.com|ip=66.189
(Question to experts: This address was never blacklisted. Messages from
that account are being scored: No, hits=-4.7 required=5.0
tests=BAYES_00,RCVD_IN_RFCI,RCVD_IN_SORBS autolearn=no version=2.63.
Hmm, no AWL? spamc calling spamd -a)
Other than that, most appear to be one-shot spams. Those that have a few
repeats are probably the result of my testing by re-feeding the message
into SA:
88.9 (88.9/1) -- *****@hotmail.com|ip=200.78
88.3 (88.3/1) -- *****@mail.com|ip=200.67
82.7 (82.7/3) -- *****@pacbell.net|ip=65.173
[...]
Do you have any rules that are adding "insanely high" scores (50, 100).
Multiple such rules?
> 56.6 (56.6/1) -- *****@earthlink.net|ip=13.111
>
>
This indicates that the SINGLE message from this user scored 56.6.
Again, that seems awfully high for a non-spam message, no matter how
badly written. I'd say an AWL false-positive for spam is unlikely. If it
was spam, that score might be perfectly valid.
> 53.7 (53.7/7) -- *****@fpl.com|ip=9.17
>
>
This addresss posted 7 messages, averaging 53.7? If they're a repeat
spammer, this is expected. If they're a regular poster to a quality
list, something must be amiss with scoring. Could they be tripping a
rule that scores very high (50+) and perhaps you're not noticing? Are
you auto-deleting high-scoring messages?
> 48.8 (48.8/163) -- *****@mercmarine.com|ip=9.17
>
>
Same here. 163 messages. All spam from same address? (Sorry, only you
can verify these, Michael. Do you archive messages, or can you search an
archive for these addresses?)
># /usr/share/doc/spamassassin/examples/check_whitelist | sort -nr | tail -n 10 | sed 's![^ ]*@!*****@!'
> -12.9 (-12.9/2) -- *****@jfet.org|ip=18.95
> -12.9 (-12.9/2) -- *****@mindspring.com|ip=65.24
>[...]
>
>
>What do you think?
>
>
Here's my output from same command:
-19.9 (-19.9/1) -- *****@io.com|ip=205.206
-19.9 (-19.9/1) -- *****@zonnet.nl|ip=205.206
[... a dozen or so similar ]
Note that these have high negative (non-spam) scores, but only ONE
message each.
-89.2 (-89.2/22) --
techbargains@topica.email-publisher.com|ip=38.113
THIS is a perfect example of what I was alluding to before. Note that I
did not obscure the full address. This is the address of the
techbargains list I subscribe to. A great little list, and all postings
to it originate from that address (it's send-only). That "insanely high"
(non-spam) score is the result of a whitelist entry that I actually
deleted a few days ago. The average for the 22 posted messages has been
-89.2. Originally, they'd have scored -100 due to whitelist (not AWL),
but they're slowly dropping to more normal ranges now without that
adjustment, thanks to the persistence of the AWL data.
Now an important detail: To be impacted by AWL, a spam message would
have to be sent FROM one of the actual auto-whitelisted addresses. Just
any message to a list won't do it. So to pull a spam message into
non-spam scoring range (<5 by default), that address would have to be
associated with "quality" messages, as shown in the outputs above.
In an earlier message in this thread, Michael D Schleif wrote:
> [...] With AWL, I get
> far too many false _negatives_, or spam labeled as ham. Without AWL I
> am getting some, a different kind of, false _negatives_, or spam labeled
> as grey. At this point, I can easier deal with the latter.
I'm not sure what you mean by "grey". Have you set some threshold for
auto-deletion that these still fall below? Messages either exceed your
spam threshold or they don't.
Messages should only be pulled towards lower (non-spam) values IF THE
SENDER ADDRESS HAS BEEN ASSOCIATED WITH GOOD POSTS IN THE PAST. Michael,
I STRONGLY suspect that your AWL data has been tainted by extremely
high-scoring rules (spam scoring, by the look of it) and that you're
seeing lingering effects in the AWL data.
This makes sense -- to the degree that I haven't seen the actual
messages -- in that if you've set up rules that score "insanely high"
(i.e. 50) and a user who's sent previous message(s) with low scores
(near 0) sends one message that trips that high scoring rule, the AWL
adjustment applied would be the negative of half the difference (i.e.
-25) which is exactly as it should be. As Keith pointed out, it will
still be scored as spam, so AWL doesn't break anything.
I DO NOT see any indication that any significant negative (non-spam)
bias is being applied by AWL currently, so your -R removals may have
cleaned up the problem. Nothing showing now should have given that -25,
nor cause spam to be tagged non-spam. However, high scoring rules may
result in seemingly high AWL adjustments if they get tripped by
known-good posters with low AWL scores.
I'd suggest you make sure you have no "insanely high" scoring individual
rules (keep them all to a few points, per the documentation, since
scoring is cumulative. I don't think scoring higher than your spam
threshold gains you much.) or black/whitelist entries. THEN try running
those same messages through again, or just monitor the affected list and
see if things have cleared up.
The hard part in realizing what's going on is only having partial
information on your specific configuration. Please be patient with folks
working on your problem. I think we've figured out the problem, and the
fixes are easy enough.
Even if you don't wind up using AWL in the end, I've certainly learned a
few things in this process!
- Bob
Re: More testing and a possible fix for skewed AWL scoring (was Re: AWL vs. mailing lists)
Posted by Michael D Schleif <md...@helices.org>.
* Bob George <ma...@ttlexceeded.com> [2004:02:26:15:07:52-0500] scribed:
> After trying to figure out Michael's problems with AWL, I've managed to get
> similarly confusing results... although I know exactly HOW they got this way...
> and how to fix 'em.
<snip />
This exposition makes more sense to me than anything else in this
thread! Thank you. I suppose, on further reflection, other data
posited will make its way into my brain as lucid information, too ;>
I use mutt, as well as other software already described; and, when I
find a -- to me -- erroneously spamassassin'ed message (among several
thousands received each day), my first reaction is to re-dispose it.
Typically, that consists of manually deploying it's de-spamassassin'ed
incarnation to its appropriate folder, and a similarly de-spamassassin'ed
instance to either folder: is.spam or not.spam. Once per night, I
process like this:
# cat ~/mail/errata/teach.spam.sh
#!/bin/sh
# *NOT* spam
sa-learn --no-rebuild --ham --mbox ~/mail/errata/.spam/not.spam
# *IS* spam
sa-learn --no-rebuild --spam --mbox ~/mail/errata/.spam/is.spam
# Re-Build database with latest expertise
sa-learn --rebuild --force-expire
exit 0
In the cases where I have attributed *WRONG* classification of spam to
AWL, I also do this:
|spamassassin -R
Please, comment on the efficacy of this process.
Much discussion on this list about AWL asserts that _only_ the sender
(From:) of the message is subject to AWL; but, I find this:
SpamAssassin auto-whitelist: removing address: tme10@lists.us.ibm.com
SpamAssassin auto-whitelist: removing address: *****@us.ibm.com
SpamAssassin auto-whitelist: removing address: owner-tme10@lists.us.ibm.com
SpamAssassin auto-whitelist: removing address: mds@helices.org
SpamAssassin auto-whitelist: removing address: OF23DD96AD.72CFFC36-ON87256E46.002C00F6-87256E46.002C00F6@us.ibm.com
These are the implicated headers:
Return-path: owner-tme10@lists.us.ibm.com
From: ***** ***** <**...@us.ibm.com>
To: tme10@lists.us.ibm.com
Message-ID: <OF...@us.ibm.com>
Sender: owner-tme10@lists.us.ibm.com
I do not know whether it gets owner-tme10@lists.us.ibm.com from
Return-path: or Sender: -- which is it? And, how does it get my
address: mds@helices.org -- the headers only us it in Received: headers?
So, clearly, AWL score is a function of many disparate data, and I
persist in my belief that voluminous mailing lists are poor applications
for AWL.
Prior to Bob's message, I could not find check_whitelist. Tonight, I
took the time, found it and used it. Please, notice that what follows
is post ad hoc spamassassin -R'ing and willy nilly adjustment of my
rules. However, let it be known that I have *NOT* adjusted black/white
lists, with the sole exception of the horrible legacy score for
amazon.com.
Please, somebody help me understand these:
# /usr/share/doc/spamassassin/examples/check_whitelist | sort -nr | head -n 50 | sed 's![^ ]*@!*****@!'
662.2 (662.2/3) -- *****@hp.com|ip=15.1
56.6 (56.6/1) -- *****@earthlink.net|ip=13.111
53.7 (53.7/7) -- *****@fpl.com|ip=9.17
52.7 (52.7/1) -- *****@ford.com|ip=8.252
52.4 (52.4/1) -- *****@apcc.com|ip=159.215
52.3 (52.3/2) -- *****@mmm.com|ip=9.17
50.8 (50.8/2) -- *****@standardlife.com|ip=9.17
50.2 (50.2/2) -- *****@nationwide.com|ip=9.17
50.1 (50.1/1) -- *****@us.ibm.com|ip=9.17
49.7 (49.7/1) -- *****@hotmail.com|ip=202.108
49.3 (49.3/1) -- *****@conrad.de|ip=80.150
49.1 (49.1/1) -- *****@jp.ibm.com|ip=9.17
49.1 (49.1/1) -- *****@ar.ibm.com|ip=9.17
49.0 (49.0/1) -- *****@aklaim.com|ip=40.231
48.8 (48.8/163) -- *****@mercmarine.com|ip=9.17
48.7 (48.7/1) -- *****@us.ibm.com|ip=9.17
48.7 (48.7/1) -- *****@us.ibm.com|ip=9.17
48.7 (48.7/1) -- *****@us.ibm.com|ip=9.17
48.7 (48.7/1) -- *****@erols.com|ip=200.44
48.5 (48.5/1) -- *****@ve.ibm.com|ip=9.17
48.5 (48.5/1) -- *****@dyadic.com|ip=9.17
47.5 (47.5/1) -- *****@laposte.net|ip=17.95
47.2 (47.2/1) -- *****@ch.ibm.com|ip=9.17
47.0 (47.0/1) -- *****@hamptonroads.com|ip=24.153
46.3 (46.3/1) -- *****@linksys.com|ip=65.96
46.3 (46.3/1) -- *****@lycos.com|ip=200.207
45.3 (45.3/1) -- *****@eniuk.com|ip=212.187
45.3 (45.3/1) -- *****@vn.pwc.com|ip=203.11
45.3 (45.3/1) -- *****@cotyinc.com|ip=212.157
45.3 (45.3/1) -- *****@kdhe.state.ks.us|ip=165.201
45.3 (45.3/1) -- *****@worldbank.org|ip=138.220
45.3 (45.3/1) -- *****@abnamro.com|ip=65.173
45.2 (45.2/2) -- *****@us.ibm.com|ip=9.17
45.1 (45.1/2) -- *****@ca.ibm.com|ip=9.56
45.1 (45.1/1) -- *****@uk.ibm.com|ip=9.17
45.1 (45.1/1) -- *****@fimat.co.uk|ip=62.189
45.1 (45.1/1) -- *****@metlife.com|ip=216.163
45.1 (45.1/1) -- *****@amadeus.net|ip=193.23
44.9 (44.9/1) -- *****@panasonic.com|ip=66.233
44.9 (44.9/1) -- *****@msn.com|ip=68.42
44.6 (44.6/1) -- *****@yahoo.com|ip=196.12
44.4 (44.4/1) -- *****@lafcs.cnyric.org|ip=81.205
44.4 (44.4/1) -- *****@hotmail.com|ip=213.132
44.1 (44.1/1) -- *****@msn.com|ip=202.163
44.0 (44.0/1) -- *****@ebay.com|ip=24.145
43.8 (43.8/1) -- *****@gaijin.co.jp|ip=81.77
43.6 (43.6/1) -- *****@acsc.net|ip=128.103
42.8 (42.8/1) -- *****@hotmail.com|ip=218.22
42.6 (42.6/1) -- *****@bkkmail.com|ip=19.63
42.4 (42.4/1) -- *****@ebay.com.au|ip=24.158
# /usr/share/doc/spamassassin/examples/check_whitelist | sort -nr | tail -n 10 | sed 's![^ ]*@!*****@!'
-12.9 (-12.9/2) -- *****@jfet.org|ip=18.95
-12.9 (-12.9/2) -- *****@mindspring.com|ip=65.24
-12.9 (-12.9/1) -- *****@delinked.us|ip=131.193
-12.9 (-12.9/1) -- *****@mit.edu|ip=221.187
-12.9 (-12.9/1) -- *****@mit.edu|ip=208.54
-12.9 (-12.9/1) -- *****@jfet.org|ip=66.18
-12.9 (-12.9/1) -- *****@jfet.org|ip=205.206
-12.9 (-12.9/1) -- *****@mail.private.network|ip=12.248
-14.9 (-14.9/5) -- *****@mail.private.network|ip=none
-14.9 (-14.9/2) -- *****@mail.private.network|ip=12.134
What do you think?
--
Best Regards,
mds
mds resource
877.596.8237
-
Dare to fix things before they break . . .
-
Our capacity for understanding is inversely proportional to how much
we think we know. The more I know, the more I know I don't know . . .
--
More testing and a possible fix for skewed AWL scoring (was Re: AWL vs. mailing lists)
Posted by Bob George <ma...@ttlexceeded.com>.
After trying to figure out Michael's problems with AWL, I've managed to get
similarly confusing results... although I know exactly HOW they got this way...
and how to fix 'em.
SETUP:
I'm running SA 2.63 (debian linux), with a system-wide bayes setup. I receive
mail on a list that is normally fine (bugtraq@securityfocus.com). Someone
recently posted there from a domain listed in a blacklist, resulting in an
overall score of 88 ("insanely spammy").
1. After realizing the problem, I've REMOVED the blacklisting from the .cf
(it's a popular .cf), and restarted spamd, the message is score dropped to
(results edited):
Content analysis details: (38.4 points, 5.0 required)
-4.9 BAYES_00 BODY: Bayesian spam probability is 0 to 1%
[score: 0.0000]
-15 USER_IN_DEF_WHITELIST From: address is in the default white-list
50 AWL AWL: Auto-whitelist adjustment
2. Noticing the AWL score adjustment, I su'ed to the receiving user, and ran
both:
spamassassin -R < msg
and
spamassassin --remove-addr-from-whitelist="address"
and it (appeared) to drop (results edited):
Content analysis details: (18.4 points, 5.0 required)
-4.9 BAYES_00 BODY: Bayesian spam probability is 0 to 1%
[score: 0.0000]
-15 USER_IN_DEF_WHITELIST From: address is in the default white-list
30 AWL AWL: Auto-whitelist adjustment
3. I restarted spamd (having thought per-user whitelist adjustments did NOT
require restart), and got:
Content analysis details: (13.4 points, 5.0 required)
-4.9 BAYES_00 BODY: Bayesian spam probability is 0 to 1%
[score: 0.0000]
-15 USER_IN_DEF_WHITELIST From: address is in the default white-list
25 AWL AWL: Auto-whitelist adjustment
I thought that indicated a restart mattered... but on further reflection, I it
doesn't. The AWL data is persistent.
4. I figured out that /usr/share/spamassassin/60_whitelist.cf contains:
def_whitelist_from_rcvd *@securityfocus.com
securityfocus.com
which I left in, but am relieved to have discovered. (Note: No similar entry
for debian!)
5. I fed the message into spamassassin without AWL, and it's scoring a negative
(non-spam) score as expected.
6. It (finally) dawned on me that spamd is doing the AWL scoring, and IT is
running as a different user. I su'ed to THAT user and did:
spamassassin --remove-addr-from-whitelist="address"
7. NOW it's working as expected:
Content analysis details: (-11.6 points, 5.0 required)
-4.9 BAYES_00 BODY: Bayesian spam probability is 0 to 1%
[score: 0.0000]
-15 USER_IN_DEF_WHITELIST From: address is in the default white-list
8. After poking about, I found check_whitelist under
/usr/share/doc/spamassassin and moved it to /usr/local/bin. I verified that the
address was gone after step 6, and that it is now back after step 7:
check_whitelist | grep domain
<other entries -- actual spammers!>
-11.6 (-11.6/1) -- address@domain|ip=X.X
So, AWL is working AS IT SHOULD, and re-testing the message WITHOUT clearing
the AWL database (for the spamassassin/spamd user) is resulting in
progressively lower scores each pass (AWL levelling out). The key to FIXING AWL
after white/blacklist problems seems to judicious use of check_whitelist
and --remove-add-from-whitelist= .
So, if AWL seems WAY off:
1. Check ALL .cf files to make sure there are no erroneous black or whitelist
entries. Remove any that are causing problems. THESE CAN CONTINUE TO CAUSE AWL
PROBLEMS EVEN LONG AFTER DELETION!
2. Verify that you're running as the user that actually does the AWL lookups
(spamassassin or spamd).
3. Take a look at the AWL database with check_whitelist.
4. Drop any "insane" scores in there with
spamassassin --remove-addr-from-whitelist=<name> (regardless of whether it was
a white OR BLACKlist entry that broke it.) Maybe just delete (good idea?) files
if they're all out-of-whack.
5. Verify they're gone with check_whitelist
6. Repeat testing (save those erroneously scored messages!)
7. Continue using AWL and enjoy the benefits it provides.
Does this seem a sound approach? Didn't find anything similar on the Wiki.
Aren't the default whitelists (which I'd previously overlooked) a potential
exploit waiting to happen? I haven't been able to exploit them, but then I've
barely tried.
In retrospect, it seems that white/blacklisting is a very heavy-handed and
error-prone approach, especially if the domains or users later change their
ways. I was actually lucky that I only had ONE user (spamd) to fix, but I can
imagine doing hundreds or thousands. A change to a SYSTEM config file can
ripple it's way into USER configurations that can be tricky to pinpoint (for
the non SA expert like myself). I think I will convert black/whitelist entries
into scored rules with "less insane" values to minimize these problems in the
future. Perhaps only enough to offset any bayes scoring adjustments.
Any thoughts or feedback?
- Bob
Re: Custom Rules
Posted by Matthias Fuhrmann <Ma...@stud.uni-hannover.de>.
On Wed, 25 Feb 2004, Matt Kettler wrote:
> At 01:19 PM 2/25/2004, rons dixon wrote:
> >rawbody LOCAL_JPG_RULE /\.jpg/i
> >score LOCAL_JPG_RULE 3.0
> >describe LOCAL_JPG_RULE
> >
> >rawbody LOCAL_GIF_RULE /\.gif/i
> >score LOCAL_GIF_RULE 3.0
> >describe LOCAL_GIF_RULE
> >
> >
> >However emails are still coming through with gif embedded in them.
>
> SA's mime hander removes any octet-stream or other binary sections before
> running any rules, even rawbody ones.
>
> Thus, you cannot create a SA rule which will match attachment filenames for
> binary types.
20_body_tests.cf has this rule:
rawbody MICROSOFT_EXECUTABLE eval:check_for_mime('microsoft_executable')
describe MICROSOFT_EXECUTABLE Message includes Microsoft executable program
so it might possible evaluating other attachments too ?!
regards,
Matthias
Re: [spa] Re: Custom Rules
Posted by Rich Puhek <rp...@etnsystems.com>.
Charles Gregory wrote:
> full LOC_DBLEXT /name="?[^"]*\.(?:html?|txt|doc|jpe?g|gif|wpd|pdf|zip)\.(?:pif|exe|com|cmd|bat|scr)/i
> describe LOC_DBLEXT Message attachment has VIRUS-style double extension
> score LOC_DBLEXT 0.5
>
> In case line-truncation ate the first line, here it is, line-wrapped:
> full LOC_DBLEXT /name="?[^"]*\.(?:html?|txt|doc|jpe?g|gi\
> f|wpd|pdf|zip)\.(?:pif|exe|com|cmd|bat|scr)/i
>
> Someone be kind enough to run this through a corpus for me?
>
> - Charles
Poor results here...
0.000 0.0000 0.0000 0.500 0.12 0.50 LOC_DBLEXT
All my email has to pass a virus scanner, though, so I suppose it's
unlikely that my corpus has any viruses in it.
It is a good sign that it didn't hit any ham, though. I believe I have
some files with (legit) attachments in there.
--Rich
Re: [spa] Re: Custom Rules
Posted by Charles Gregory <cg...@hwcn.org>.
On Wed, 25 Feb 2004, Matt Kettler wrote:
> SA's mime hander removes any octet-stream or other binary sections before
> running any rules, even rawbody ones.
> Thus, you cannot create a SA rule which will match attachment filenames for
> binary types.
As noticed earlier today, using the FULL test *does* permit scanning of
attachment file names, hence, the following works:
full LOC_DBLEXT /name="?[^"]*\.(?:html?|txt|doc|jpe?g|gif|wpd|pdf|zip)\.(?:pif|exe|com|cmd|bat|scr)/i
describe LOC_DBLEXT Message attachment has VIRUS-style double extension
score LOC_DBLEXT 0.5
In case line-truncation ate the first line, here it is, line-wrapped:
full LOC_DBLEXT /name="?[^"]*\.(?:html?|txt|doc|jpe?g|gi\
f|wpd|pdf|zip)\.(?:pif|exe|com|cmd|bat|scr)/i
Someone be kind enough to run this through a corpus for me?
- Charles
Re: Custom Rules
Posted by Matt Kettler <mk...@evi-inc.com>.
At 01:19 PM 2/25/2004, rons dixon wrote:
>rawbody LOCAL_JPG_RULE /\.jpg/i
>score LOCAL_JPG_RULE 3.0
>describe LOCAL_JPG_RULE
>
>rawbody LOCAL_GIF_RULE /\.gif/i
>score LOCAL_GIF_RULE 3.0
>describe LOCAL_GIF_RULE
>
>
>However emails are still coming through with gif embedded in them.
SA's mime hander removes any octet-stream or other binary sections before
running any rules, even rawbody ones.
Thus, you cannot create a SA rule which will match attachment filenames for
binary types.
Custom Rules
Posted by rons dixon <ro...@chemweek.com>.
RH 9.0 with CommuniGate Pro 4.1.8
I am trying to create custom user rules. I added allow_user_rules 1 the
local.cf file located /etc/mail/spamassasin/ which should now allow me =
to
create custom rules for each user. However i created the rule listed =
below
to score 3.0 points on anyemails that has a gif or jpg file embedded in =
the
body of the email.
rawbody LOCAL_JPG_RULE /\.jpg/i
score LOCAL_JPG_RULE 3.0
describe LOCAL_JPG_RULE
rawbody LOCAL_GIF_RULE /\.gif/i
score LOCAL_GIF_RULE 3.0
describe LOCAL_GIF_RULE
However emails are still coming through with gif embedded in them.
Your help will be greatly appreciated.
Thanks.
RE: AWL vs. mailing lists
Posted by Gary Funck <ga...@intrepid.com>.
> From: Michael D. Schleif [mailto:mds@bragi.private.network]On Behalf Of
> Michael D Schleif
> Sent: Wednesday, February 25, 2004 9:17 AM
[...]
>
> High volume mailing lists (e.g., debian-user) have an extremely high
> incidence of ham; therefore, AWL is extremely high for such lists. When
> the virus/worm-of-the-day spams the list, SA identifies these as spam,
> then transforms them into ham -- solely due to AWL skewing ;<
>
> I know that I can turn OFF AWL.
>
> Is that my only alternative?
>
> What do you think?
1. For viruses, a virus scanner is the better tool.
2. Be warned though, on techie mail lists it can be quite okay to send
a .bat file, a .bas file, and even the occasional .exe file. Therefore,
quaranteening is a better policy than outright deleting 'dangerous'
attachments that can't be directly determined to contain a known virus.
3. You could drop auto-whitelisting, and use 'more_spam_to
mailinglist@frob.com'.
Personally, apart from mailing lists like spamassassin, spam-l, and procmail
where spam is often a common topic, I've found little loss from just filing
suspected spam away in the "possible spam" file. I'm also not a big fan
of auto-whitelisting these days.