Question on focus of Bayes

[Theodore Heise <th...@heise.nu>]

Hi all,

I don't recall seeing discussion of this question before, but 
apologize if I've missed it.

Occasionally I get unsolicited bulk e-mail on a topic that is of 
borderline interest to me.  My tendency is to deleted it from my 
spam folder before training the Bayes functions on my spam.  I've 
considered training Bayes on these messages as ham, but I don't know 
if that might have a tendency to lower scores on unsolicited bulk 
e-mail in general.

I guess what I'm wondering about is the relative weights of the 
headers and other evidence that a message was a bulk mailing as 
compared to the message content itself.

I'm not sure I've even asked the question clearly enough, but hope 
I've gotten the idea across.  Any feedback would be appreciated.

-- 
Theodore (Ted) Heise     <th...@heise.nu>     Bloomington, IN, USA

Re: Question on focus of Bayes

[Matus UHLAR - fantomas <uh...@fantomas.sk>]

On 09.03.08 14:15, Theodore Heise wrote:
> Occasionally I get unsolicited bulk e-mail on a topic that is of 
> borderline interest to me.  My tendency is to deleted it from my 
> spam folder before training the Bayes functions on my spam.  I've 
> considered training Bayes on these messages as ham, but I don't know 
> if that might have a tendency to lower scores on unsolicited bulk 
> e-mail in general.
> 
> I guess what I'm wondering about is the relative weights of the 
> headers and other evidence that a message was a bulk mailing as 
> compared to the message content itself.

I don't think that should be a problem. If you feel that the message is a
spam, train it as a spam, if you think it's ham (even of a low interest to
you), train it as ham. If you are not sure, you don't need to train at all
(unless you think it's FP/FN).

If you have subscribed to mailing lists, you should not train messages sent
to it as spam, care of that should be done on the list level. The same
applies to mailings from companies that have your address because of some
(valid) reason. You can still try to unsubscribe...

I think that proper training should spoil anything. I already trained quite
well Bayes DB on abuse@ mailbox, which is a place where many reports come.
-- 
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
10 GOTO 10 : REM (C) Bill Gates 1998, All Rights Reserved!

Re: Question on focus of Bayes

[Kris Deugau <kd...@vianet.ca>]

Alex Woick wrote:
> If you feed third-party backscatter as spam, you let the Bayes filter 
> know that the standard bounce texts like "...unable to deliver..." is a 
> spam indicator.

... but if you feed in a few legitimate bounces as ham, the bounce text 
itself tends towards a neutral rating over time.  The returned message 
ends up being the primary factor in Bayes scan results.

*shrug*  Works For Me.

> For better backscatter handling, look into the VBounce plugin of 
> SpamAssassin. This is the short description:

*nod*  I'm aware of that plugin (or ruleset, pre-3.1?).  I've had no 
need of it so far because Bayes has been doing a good enough job.

-kgd

Re: Question on focus of Bayes

[Alex Woick <al...@wombaz.de>]

>> The proper usage of the Bayes filter is very simple: feed spam as spam 
>> and ham as ham. All of your mail. Don't care for content that might be 
>> mis-learned in your eyes: it will not be mis-learned. Don't try be 
>> smarter than the filter. The only exception is bounce-messages: don't 
>> feed them at all.
> 
> Why not?  I've found that to be a wonderful way to discard third-party 
> backscatter without dropping the legitimate notices regarding mail my 
> customers actually sent (to the wrong place, mind you...)

If you feed third-party backscatter as spam, you let the Bayes filter 
know that the standard bounce texts like "...unable to deliver..." is a 
spam indicator. If some day you get a real bounce of a legitimate mail 
you wrote, it may end up in the spam folder as FP. On the other hand, if 
you learn bounces as ham, you mistrain the filter on the spam words, if 
it was a spam mail that bounced. So the advice to not learn bounces at all.

For better backscatter handling, look into the VBounce plugin of 
SpamAssassin. This is the short description:

# If you use this, set up procmail or your mail app to spot the
# "ANY_BOUNCE_MESSAGE" rule hits in the X-Spam-Status line, and move
# messages that match that to a 'vbounce' folder.
#
# You should also add 'whitelist_bounce_relays' lines, describing the 
names of
# your own outgoing mail relays, like so:
#
#   whitelist_bounce_relays       dogma.boxhost.net
#
# This is used to 'rescue' legitimate bounce messages that were generated in
# response to mail you really *did* send.  If you don't do this, the
# "BOUNCE_MESSAGE" rule will not fire.  See 'perldoc VBounce.pm' for more
# details.


It is activated via /etc/mail/spamassassin/v320.pre.

Re: Question on focus of Bayes

[Kris Deugau <kd...@vianet.ca>]

Alex Woick wrote:
> The proper usage of the Bayes filter is very simple: feed spam as spam 
> and ham as ham. All of your mail. Don't care for content that might be 
> mis-learned in your eyes: it will not be mis-learned. Don't try be 
> smarter than the filter. The only exception is bounce-messages: don't 
> feed them at all.

Why not?  I've found that to be a wonderful way to discard third-party 
backscatter without dropping the legitimate notices regarding mail my 
customers actually sent (to the wrong place, mind you...)

The only things I don't feed to Bayes are very large messages (since 
those will fly past SA based on size alone, and the processing time 
isn't worth the relatively small benefit) and mail that I *can't* 
identify.  (Yes, I *have* met some.  O_o  Yick.)

Results since I upgraded my personal system to 2.53 (IIRC) and the ISP 
systems I maintained at the time to 2.54 have been pretty darn good, IMO.

Way better than Postini seems to be doing these days.  <shudder>

-kgd

Re: Question on focus of Bayes

[maillist <ma...@emailacs.com>]

Theodore Heise schrieb am 09.03.2008 19:15:
>
>> Occasionally I get unsolicited bulk e-mail on a topic that is of 
>> borderline interest to me.  My tendency is to deleted it from my spam 
>> folder before training the Bayes functions on my spam.  I've 
>> considered training Bayes on these messages as ham, but I don't know 
>> if that might have a tendency to lower scores on unsolicited bulk 
>> e-mail in general.

If this is the case, then why filter mail at all?  What if you miss 
something that you may want?

Seriously though, that's just crazy talk.  If you didn't request the 
mail in some form or fashion, then it's SPAM, bottom-line.  Do I want to 
see real photos of Angelina Jolie nude - hell yeah! - but it's still 
spam, and should be learned as such.

-Aubrey

Re: Question on focus of Bayes

[Alex Woick <al...@wombaz.de>]

Theodore Heise schrieb am 09.03.2008 19:15:

> Occasionally I get unsolicited bulk e-mail on a topic that is of 
> borderline interest to me.  My tendency is to deleted it from my spam 
> folder before training the Bayes functions on my spam.  I've considered 
> training Bayes on these messages as ham, but I don't know if that might 
> have a tendency to lower scores on unsolicited bulk e-mail in general.

The proper usage of the Bayes filter is very simple: feed spam as spam 
and ham as ham. All of your mail. Don't care for content that might be 
mis-learned in your eyes: it will not be mis-learned. Don't try be 
smarter than the filter. The only exception is bounce-messages: don't 
feed them at all.

There are always more spam-indicators in a spam mail than you think of 
in the first place. If you learn a spam mail as ham, you mistrain the 
filter on these indicators. If you learn this spam (with content you are 
interested in) as spam, and you also learn ham messages of similar 
content, the words that are contained in both messages are less 
considered a spam/ham sign any more. So these words don't push the 
detection in the wrong direction. They simply push in no direction. But 
there are always enough other words left that push the mails in the 
correct direction. Not only the message body is fed into the database, 
but also some headers.

Tschau
Alex