You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Charles Gregory <cg...@hwcn.org> on 2009/12/17 17:52:04 UTC

Re: [sa] Re: emailreg.org - tainted white list

On Thu, 17 Dec 2009, Yet Another Ninja wrote:
> On 12/16/2009 6:16 PM, Charles Gregory wrote:
>>  On Wed, 16 Dec 2009, Yet Another Ninja wrote:
>> >  blabber... checkout SVN - follow dev list... HABEAS is history...
>>  I believe the *point* here is that HABEAS is NOT 'history' for ordinary
>>  systems running ordinary sa-update on 3.2.5.....
>
> they can adjust scores if they don't approve of what has been delivered...

Agreed. But that does not make the statement "HABEAS is history" accurate 
in any way that is relevant to current sa-update......

>>  My rules (in /var/lib/spamassassin) still include the strong negative
>>  scores for HABEAS, as discussed here.
> funny.. my rules show a 0 score for HABEAS stuff, same with all the other 
> "certification services"  oh wait!! I adjusted the scores myself coz I didn't 
> want them in my way.

Why don't you go one step further and just 'unsubscribe' from any spam you 
receive? If you want the ultimate in responsive after-the-spam-has-arrived 
customization, that's the way to go.... ;)

Oh. Sorry. Someimes the sarcasm gets away from me.

We are discussing the DEFAULT rules. The only way someone can tell me that 
HABEAS is "history" and have it apply to ME is if they have propogated a 
change through sa-update. They haven't. Your customizatino sounds a lot 
like mine. But just because you and I have solved our problems for *us* 
personally does not mean we can just forget about everyone else.

You're a Ninja, judging by your From header. You *must* be in this to 
improve things for everyone. I'm certainly not posting here just to hear 
myself talk. I can customize my server far faster (it's actually a daily 
routine) than I can type suggestions here. But I want this to work for 
everyone. And everyone is not on this list. So changing SA defaults is the 
best way to help everyone.

I don't have the 'budget' to just jump in and help code, so I make 
suggestions, with (I hope) the appropriate tone of respect for the people 
who *do* have the 'budget' to be working on improving SA. But this is NOT 
me whining about *my* problems. I don't have a problem with HABEAS. I 
occasionally notice their rule fire, but usually something else knocks 
out the spam anyways.... (shrug)

- C

Re: [sa] Re: habeas - tainted white list

Posted by jdow <jd...@earthlink.net>.
From: "R-Elists" <li...@abbacomm.net>
Sent: Thursday, 2009/December/17 11:21
>
>> I believe on the whole Warren Togami's posting about a
>> whitelist performance on a masscheck settles the affair.
>> White lists are very reliable. They are also very unnecessary
>> within SpamAssassin. So perhaps the whole topic can die.
>>
>> I also note that the people complaining about the white lists
>> seem to leave out solid data. Were the "spams" really
>> confirmed spams or were they merely scored as spams? What
>> scores hit that made them score as spams? What kind of
>> installation do you have? How many emails a day are processed?
>>
>> It's little details like that which prompt other people to
>> look at assertions somewhat askance or ignore them outright.
>>
>> With my three personal accounts I have yet to see an email
>> off this list containing HABEAS, spam or ham, since this
>> discussion began. I guess I don't do business with HABEAS
>> customers and no spammers have pushed through anything from a
>> HABEAS site. The mail volume is fairly high (LKML and a
>> couple other Linux lists). And the spam seems to be suddenly
>> up from 60-80 a day to the 90s/day. For those spammers who
>> are listening, I REALLY do not need Via-thingie-alis whether
>> or not it is from he Pf people. If I REALLY need to get it up
>> I do a sexy striptease or something like that. (The V thingie
>> seems to be a new feature of my spam bucket - 10 or more of
>> them a day.)
>>
>> {^_-}
>>
>
> JDow et al,
>
> why do you say "on the whole" ? what is holding you back in your thinking
> there?
>
> ...based upon Togami's data processing, the biggest thing that comes to 
> mind
> is this...
>
> *IF* these or similar rulesets are not truly not making a difference one 
> way
> or the other, then why are they there?
>
> why do we really need them or the other similar rulesets?
>
> ...and why should any rules "such as these" have a default SA installation
> value other than "zero" and then educate admins in the documentation what 
> to
> do in regards to enabling and suggested scoring?

I read Warren's note to indicate their scores were being made sensible
in line with what the masscheck indicates. If they are 100% effective and
only 1% needed the score would be very low despite the accuracy. That makes
sense as a starting point. Then it's up to the administrators to put in
their custom rules to account for effects like "one person's spam is
another person's ham," and "I don't want to bother to unsubscribe, I'll
just declare this list spam."

The tools might be good as an SMTP transaction time test, though. Use a
positive hit as a gateway through the greylisting wall, perhaps. It might
put a small fraction of a percent more load on SpamAssassin. But it might
be worthwhile.

Heck, I'm only administering a two person net here and I take the time
to learn the tools I am using and write useful configurations for them.
Somebody paid to do this should do no less. Otherwise, do something
silly and purchase a Barracuda if the boss is too dumb to pay you to
do it right.

{^_^} 


Re: [sa] Re: habeas - tainted white list

Posted by Christian Brel <br...@copperproductions.co.uk>.
On Fri, 18 Dec 2009 13:21:00 -0500 (EST)
Charles Gregory <cg...@hwcn.org> wrote:

> On Fri, 18 Dec 2009, Christian Brel wrote:
> > There comes a time when you need to deal with that and move on. We
> > are all grown up now and not - like you say - '5 & 6 year old
> > children'.
> 
> Good. Then stop talking like them.
Perhaps you need to stop *acting* like them ;-)
> 
> > Please feel free to act like an adult and end the personal attacks,
> > or, act like a troll. It's your reputation ;-)
> 
> The man who got banned and had to fake a new user name is lecturing
> me on reputation? ROFLMAOUIPMP
> 
So two wrongs would make a right. I see. Yep, I'm laughing too :-)

> > Return Path:
> > "Today we are the world’s leading email deliverability services
> > company and our clients include Fortune 500 firms"
> 
> There. You now have the answer to your question. So stop asking it.
> (Finally)
I don't thing anyone was ever under the impression they were a charity
doing it for love. But that would be an assumption. After all, those
HABEAS 'oil can' rules are in Spamassassin for love and not money....

> 
> > do you think this is a commercial enterprise or a charity?
> 
> Do I think you will ever ask any questions not already answered or
> obvious from the website?
> 
> - C
I apologise, that was rude of me. I was told *not* to assume something
even if it was obvious. So it's clear for the Archives;

Return Path is a commercial operation that makes money.
Return Path mail is eased through Spamassassin with negative scoring
rules.
Asking if any money changed hands for this position of privilege
provokes hostility.
Despite these rules benefiting the commercial interests of Return Path,
and not necessarily the users - and despite there being no fiscal
reward for Apache/Spamassassin - this state of affairs will remain.

Yep, I'm clear on that.

Most of this has been addressed by Daryl in grown up talk whilst you
were tucked up in your bed.

I would like to take this opportunity to thank you Charles, you've
really made me laugh this afternoon and I love you. X X X. You've been
really helpful and I'm glad you've become my friend :-) Have a Merry
Christmas.
-- 
This e-mail and any attachments may form pure opinion and may not have
any factual foundation. Please check any details provided to satisfy
yourself as to suitability or accuracy of any information provided.
Data Protection: Unless otherwise requested we may pass the information
you have provided to other partner organisations. 

Re: [sa] Re: habeas - tainted white list

Posted by Charles Gregory <cg...@hwcn.org>.
On Fri, 18 Dec 2009, Christian Brel wrote:
> There comes a time when you need to deal with that and move on. We are
> all grown up now and not - like you say - '5 & 6 year old children'.

Good. Then stop talking like them.

> Please feel free to act like an adult and end the personal attacks, or,
> act like a troll. It's your reputation ;-)

The man who got banned and had to fake a new user name is lecturing me on 
reputation? ROFLMAOUIPMP

> Return Path:
> "Today we are the world’s leading email deliverability services company
> and our clients include Fortune 500 firms"

There. You now have the answer to your question. So stop asking it.
(Finally)

> do you think this is a commercial enterprise or a charity?

Do I think you will ever ask any questions not already answered or obvious 
from the website?

- C

Re: habeas - tainted white list

Posted by Christian Brel <br...@copperproductions.co.uk>.
On Fri, 18 Dec 2009 12:03:38 -0500 (EST)
Charles Gregory <cg...@hwcn.org> wrote:

> On Fri, 18 Dec 2009, Christian Brel wrote:
> >>> You need to resort to abuse for what particular reason?
> >> Repeatedly accusing the SA developers of fraudulent collusion is
> >> abusive. Don't be surprised if people are abusive in return.
> > That is your choice of words - not mine. It is interesting that
> > when reasonable questions about the motivation for a bizarre part
> > of SA is brought up, others are entitled to abuse the person with
> > that point of view - but he must not respond to that abuse or runs
> > the risk of the mob ganging up.
> 
> Now where have I heard this before...?   Sounds so familiar.....
> 
> Ah! Right! Got it.....
> My (then) 5 and 6 year old children arguing over who "started it".
> 
> - C
> PS. You did. No one calls you 'troll' until you act like one.

And this pointless post you have just made is ?not? trolling to provoke
a reaction? I apologise if at some point in the past I've hurt your
feelings or made you look small. Sincerely.

There comes a time when you need to deal with that and move on. We are
all grown up now and not - like you say - '5 & 6 year old children'.

Please feel free to act like an adult and end the personal attacks, or,
act like a troll. It's your reputation ;-)

BTW:
Return Path:
"Today we are the world’s leading email deliverability services company
and our clients include Fortune 500 firms" do you think this is a
commercial enterprise or a charity?


-- 
This e-mail and any attachments may form pure opinion and may not have
any factual foundation. Please check any details provided to satisfy
yourself as to suitability or accuracy of any information provided.
Data Protection: Unless otherwise requested we may pass the information
you have provided to other partner organisations. 

Re: habeas - tainted white list

Posted by John Hardin <jh...@impsec.org>.
On Fri, 18 Dec 2009, Christian Brel wrote:

> On Fri, 18 Dec 2009 06:19:25 -0800 (PST)
> John Hardin <jh...@impsec.org> wrote:
>
>> We understand your philosophical objection. Providing hard evidence
>> of FNs will go much further towards making your point than name
>> calling will.
>
> The name calling being?

Alright, let me amend that: Providing hard evidence of FNs will go much 
further towards making your point - and getting the rules fixed in a 
useful manner - than will repeated accusations that the SA devs are taking 
bribes to weaken SA.

And phrasing it as a question doesn't make it any less of an accusation, 
given it keeps being repeated after reasonable explanations have been 
provided.

At the moment there's insufficient _hard data_ to support the contention 
that the reputation whitelists are assisting FNs to a great degree. The 
data from masscheck suggests the impact of the reputation whitelists is 
neutral to very slightly positive (in terms of reducing FPs). If you feel 
this isn't justified, if you're seeing a lot of FNs that can be laid at 
the feet of a reputation whitelist rule, then please feed that hard data 
into the masscheck corpora so that the scoring process can take it into 
account.

-- 
  John Hardin KA7OHZ                    http://www.impsec.org/~jhardin/
  jhardin@impsec.org    FALaholic #11174     pgpk -a jhardin@impsec.org
  key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
   "Bother," said Pooh as he struggled with /etc/sendmail.cf, "it never
   does quite what I want. I wish Christopher Robin was here."
 				           -- Peter da Silva in a.s.r
-----------------------------------------------------------------------
  7 days until Christmas

Re: habeas - tainted white list

Posted by Charles Gregory <cg...@hwcn.org>.
On Fri, 18 Dec 2009, Christian Brel wrote:
>>> You need to resort to abuse for what particular reason?
>> Repeatedly accusing the SA developers of fraudulent collusion is
>> abusive. Don't be surprised if people are abusive in return.
> That is your choice of words - not mine. It is interesting that  when
> reasonable questions about the motivation for a bizarre part of SA is
> brought up, others are entitled to abuse the person with that point of
> view - but he must not respond to that abuse or runs the risk of the
> mob ganging up.

Now where have I heard this before...?   Sounds so familiar.....

Ah! Right! Got it.....
My (then) 5 and 6 year old children arguing over who "started it".

- C
PS. You did. No one calls you 'troll' until you act like one.

RE: habeas - tainted white list

Posted by R-Elists <li...@abbacomm.net>.
 

> 
> Spamassassin is not something trivially installed like a 
> piece of Microsoft junkware. In fact, it is nearly impossible 
> to get it to do anything useful without reading lots of 
> documents Daryl. Couple this with the fact it only *scores* 
> mail - it does not block it - any mish mash of rules could be 
> argued to be 'safe'. If it were deployed at the SMTP level 
> where it was kicking out 55x's it may be a different story.
> So the 'safe' angle really has no legs.
> 

CB,

the thing is, some of us do have SA integrated in such a way as to reject
spam at SMTP time

that is one of the main reasons why we do not believe that UBE should be
given preference in SA as a *default*

in general, if legit companies with legit MOI lists and whatever other
*legit* email admin concerns cannot make simple emails that will not be
scored as spam and make into easily into people's email boxes then too bad.

way too much & way to frequent UBE junk.

 - rh


Re: habeas - tainted white list

Posted by Christian Brel <br...@copperproductions.co.uk>.
On Fri, 18 Dec 2009 10:33:31 +0100
Benny Pedersen <me...@junc.org> wrote:

> On fre 18 dec 2009 10:23:48 CET, Christian Brel wrote
> 
> >> If you like you can transparently disable the DNSWLs.
> > I found it much more useful to apply them as blocklists and give
> > the a +4/+8 myself - but that's a personal choice.
> 
> and "No, hits=0.7 required=10.0 tests=SPF_SOFTFAIL" is also a
> personal choice ?
> 
For what I am doing, yes ;-)

-- 
This e-mail and any attachments may form pure opinion and may not have
any factual foundation. Please check any details provided to satisfy
yourself as to suitability or accuracy of any information provided.
Data Protection: Unless otherwise requested we may pass the information
you have provided to other partner organisations. 

Re: habeas - tainted white list

Posted by Benny Pedersen <me...@junc.org>.
On fre 18 dec 2009 10:23:48 CET, Christian Brel wrote

>> If you like you can transparently disable the DNSWLs.
> I found it much more useful to apply them as blocklists and give the a
> +4/+8 myself - but that's a personal choice.

and "No, hits=0.7 required=10.0 tests=SPF_SOFTFAIL" is also a personal  
choice ?

-- 
xpoint http://www.unicom.com/pw/reply-to-harmful.html

Re: habeas - tainted white list

Posted by Christian Brel <br...@copperproductions.co.uk>.
On Fri, 18 Dec 2009 04:07:55 -0500
"Daryl C. W. O'Shea" <sp...@dostech.ca> wrote:

> > If everything is open and transparent give the default user the
> > option to *enable* them and score them zero, unless - of course -
> > there is some kind of logical reason for these mad scoring spam
> > assisting rules that favour Return Path in the default set up?
> 
> I stand firm on my opinion that our principle of safe for most users
> is the logical reason for including DNSWLs.

Spamassassin is not something trivially installed like a piece of
Microsoft junkware. In fact, it is nearly impossible to get it to do
anything useful without reading lots of documents Daryl. Couple this
with the fact it only *scores* mail - it does not block it - any mish
mash of rules could be argued to be 'safe'. If it were deployed at the
SMTP level where it was kicking out 55x's it may be a different story.
So the 'safe' angle really has no legs.


> 
> If you like you can transparently disable the DNSWLs.
I found it much more useful to apply them as blocklists and give the a
+4/+8 myself - but that's a personal choice.

Thank you for your time Daryl. We don't agree - but I don't want to
waste more of your personal time on this.


-- 
This e-mail and any attachments may form pure opinion and may not have
any factual foundation. Please check any details provided to satisfy
yourself as to suitability or accuracy of any information provided.
Data Protection: Unless otherwise requested we may pass the information
you have provided to other partner organisations. 

Re: habeas - tainted white list

Posted by Charles Gregory <cg...@hwcn.org>.
On Fri, 18 Dec 2009, Christian Brel wrote:
> Why not default them to zero and include in the release notes/man that
> there are whitelists and they can *enable* them?

Go read the archives, troll.

- C

Re: [sa] Re: habeas - tainted white list

Posted by "Daryl C. W. O'Shea" <sp...@dostech.ca>.
On 18/12/2009 4:46 PM, Charles Gregory wrote:
> On Fri, 18 Dec 2009, jdow wrote:
>> I suppose it's not a whole lot of bother to change the 3.2 scores.
>> But, people who feel they have been bitten with a HABEAS score have
>> probably already overridden them.
> 
> Again, I make a note that my concern is for the thousands who install a
> 'pre-canned' Spamassassin install, with a wrapper to handle what happens
> to the messages, etc, etc. If you feel a slight chill at the notion of
> people operating mail servers with so little knowledge, I'm right there
> with you, but I *was* one of these people a few years ago. Stumbling and
> learning. Trial by fire. Fun way to learn. :)

Interestingly this is one of the reasons why we err on the side of
not-tagging mail.

Daryl


Re: [sa] Re: habeas - tainted white list

Posted by jdow <jd...@earthlink.net>.
From: "Charles Gregory" <cg...@hwcn.org>
Sent: Friday, 2009/December/18 13:46


> On Fri, 18 Dec 2009, jdow wrote:
>> I suppose it's not a whole lot of bother to change the 3.2 scores. But, 
>> people who feel they have been bitten with a HABEAS score have probably 
>> already overridden them.
>
> Again, I make a note that my concern is for the thousands who install a 
> 'pre-canned' Spamassassin install, with a wrapper to handle what happens 
> to the messages, etc, etc. If you feel a slight chill at the notion of 
> people operating mail servers with so little knowledge, I'm right there 
> with you, but I *was* one of these people a few years ago. Stumbling and 
> learning. Trial by fire. Fun way to learn. :)
>
> So the more that can be 'standardized' without jeopardizing flexibility, 
> the better things can be.... :)
>
>>>  If you like you can transparently disable the DNSWLs.
>> Is he smart enough to do so?
>
> With out regard for who 'he' is, it is certain that *someone* out there is 
> not that 'smart', and follows the 'recommendations' provided by their 
> hosting provider for a 'standard' mail server setup. They will just want 
> it to 'work' without any maintenance at all.
>
> And just to beat out the next inevitable argument, no, these people are 
> not 'lazy'. They just literally don't know what they are doing. If someone 
> doesn't pre-build the system properly, they end up running open relays.
> Yes, THOSE people..... :(

Once 3.3 is out the problem is solved if they have a distro that reviews
and updates the packages it distributes. (Yes, that IS a big if, as with
regards to Fedora and ClamAV. {^_-}) If SpamAssassin is not updated what
makes you think the distro would have the automatic updates for the rules
enabled? I just don't see SpamAssassin as a suitable tool for a person
who is a perfectionist and not a tinkerer. (No tool is suitable for
such a person, for that matter.)

Updating 3.2 is probably not as important as getting 3.3 out. And given
the few number of complaints updating 3.2 is likely quite the opposite
of critical. Look how long it's been out before it took a nutcase to
start complaining leading to the discovery of this alleged problem.
(Even the respected Lukreme has not stated outright that the item for
which he showed scores was really confirmed spam as opposed to a
disgruntled user trying to get off a mailing list and not willing to
follow simple instructions for doing so.)

{o.o} 


Re: [sa] Re: habeas - tainted white list

Posted by Charles Gregory <cg...@hwcn.org>.
On Fri, 18 Dec 2009, jdow wrote:
> I suppose it's not a whole lot of bother to change the 3.2 scores. But, 
> people who feel they have been bitten with a HABEAS score have probably 
> already overridden them.

Again, I make a note that my concern is for the thousands who install a 
'pre-canned' Spamassassin install, with a wrapper to handle what happens 
to the messages, etc, etc. If you feel a slight chill at the notion of 
people operating mail servers with so little knowledge, I'm right there 
with you, but I *was* one of these people a few years ago. Stumbling and 
learning. Trial by fire. Fun way to learn. :)

So the more that can be 'standardized' without jeopardizing flexibility, 
the better things can be.... :)

>>  If you like you can transparently disable the DNSWLs.
> Is he smart enough to do so?

With out regard for who 'he' is, it is certain that *someone* out there is 
not that 'smart', and follows the 'recommendations' provided by their 
hosting provider for a 'standard' mail server setup. They will just want 
it to 'work' without any maintenance at all.

And just to beat out the next inevitable argument, no, these people are 
not 'lazy'. They just literally don't know what they are doing. If someone 
doesn't pre-build the system properly, they end up running open relays.
Yes, THOSE people..... :(

- C

Re: habeas - tainted white list

Posted by jdow <jd...@earthlink.net>.
From: "Daryl C. W. O'Shea" <sp...@dostech.ca>
Sent: Friday, 2009/December/18 01:07
> On 18/12/2009 3:32 AM, Christian Brel wrote:
>> On Fri, 18 Dec 2009 02:24:45 -0500
>> "Daryl C. W. O'Shea" <sp...@dostech.ca> wrote:
...
>>> From the data we have from mass-checks we are erring a very small
>>> amount on the side of caution by not disabling the whitelists by
>>> default.
>> It's a big fat favourable score to one organisation for 'erring a very
>> small amount on the side of caution' don't you think? -4/-8 given the
>> average 419 spam only scores 4-8 points.
> 
> Again, we agree.  We've changed it in the upcomming release and will
> surely backport it when we're done getting 3.3 out.  It's been like this
> for years, I don't think we need to jump like crazy to change the 3.2
> updates before we've even settled on a final score.

I suppose it's not a whole lot of bother to change the 3.2 scores. But,
people who feel they have been bitten with a HABEAS score have probably
already overridden them.

>> If everything is open and transparent give the default user the option
>> to *enable* them and score them zero, unless - of course - there is
>> some kind of logical reason for these mad scoring spam assisting rules
>> that favour Return Path in the default set up?
> 
> I stand firm on my opinion that our principle of safe for most users is
> the logical reason for including DNSWLs.

Indeed, HE is not the boss.

> If you like you can transparently disable the DNSWLs.

Is he smart enough to do so?

{^_^}

RE: habeas - tainted white list

Posted by Benny Pedersen <me...@junc.org>.
On Fri 18 Dec 2009 07:42:55 PM CET, R-Elists wrote
>> or create a bug to have dnswl use trusted_networks from
>> local.cf in spamassassin
> can you help me / us better understand what you are getting at here and why?

example:

trusted_networks 127.128.0.0/16

and then if 127.128.128.128 is listed in dnswl, make a rbl test that  
use firsttrusted to match it is remote listed in dnswl also, that  
means you agree its a whitelist ip, so if dnswl make some ip  
whitelisted, and its not in local.cf as trusted_networks it would not  
help you :)

> something you already do or implement?

i currently not have the need to do it, but it is supported imho

> i wish i knew a better way to ask the question(s) so that you could better
> help us understand your thinking

i could tell more about cpm, not funny ? :)

nope, its just the OT thread i am inspired of, why none of them use  
perldoc more then fighting here about something that its easely fixed  
in local.cf

-- 
xpoint http://www.unicom.com/pw/reply-to-harmful.html

RE: habeas - tainted white list

Posted by R-Elists <li...@abbacomm.net>.
 

> 
> or create a bug to have dnswl use trusted_networks from 
> local.cf in spamassassin
> 

Benny

can you help me / us better understand what you are getting at here and why?

something you already do or implement?

i wish i knew a better way to ask the question(s) so that you could better
help us understand your thinking

tia

 - rh


Re: habeas - tainted white list

Posted by Matthias Leisi <ma...@leisi.net>.
dnswl.org does offer trusted_networks-formatted files (separated by our trust levels), but beware of bug 5931 for older versions of SA: https://issues.apache.org/SpamAssassin/show_bug.cgi?id=5931

-- Matthias
 
Am 18.12.2009 um 10:17 schrieb Benny Pedersen:

> On fre 18 dec 2009 10:07:55 CET, "Daryl C. W. O'Shea" wrote
>> If you like you can transparently disable the DNSWLs.
> 
> or create a bug to have dnswl use trusted_networks from local.cf in spamassassin
> 
> -- 
> xpoint http://www.unicom.com/pw/reply-to-harmful.html


Re: habeas - tainted white list

Posted by Benny Pedersen <me...@junc.org>.
On fre 18 dec 2009 10:07:55 CET, "Daryl C. W. O'Shea" wrote
> If you like you can transparently disable the DNSWLs.

or create a bug to have dnswl use trusted_networks from local.cf in  
spamassassin

-- 
xpoint http://www.unicom.com/pw/reply-to-harmful.html

Re: habeas - tainted white list

Posted by Christian Brel <br...@copperproductions.co.uk>.
On Fri, 18 Dec 2009 06:12:06 -0800 (PST)
John Hardin <jh...@impsec.org> wrote:

> On Fri, 18 Dec 2009, Christian Brel wrote:
> 
> > On Fri, 18 Dec 2009 02:29:56 -0700
> > LuKreme <kr...@kreme.com> wrote:
> >
> >> I might agree with some small portion of our resident troll's
> >> posts,
> >
> > You need to resort to abuse for what particular reason?
> 
> Repeatedly accusing the SA developers of fraudulent collusion is
> abusive. Don't be surprised if people are abusive in return.
> 

That is your choice of words - not mine. It is interesting that  when
reasonable questions about the motivation for a bizarre part of SA is
brought up, others are entitled to abuse the person with that point of
view - but he must not respond to that abuse or runs the risk of the
mob ganging up.

It seems that *some* can alter subject lines to abuse, send abusive
off-list mail, openly abuse etc, whilst others just have to sit and
take it. When they are not happy to do that they are accused of
trolling. Strikes me as cyber-bulling, but I've no intention of rising
to it - it's all rather boring.

-- 
This e-mail and any attachments may form pure opinion and may not have
any factual foundation. Please check any details provided to satisfy
yourself as to suitability or accuracy of any information provided.
Data Protection: Unless otherwise requested we may pass the information
you have provided to other partner organisations. 

Re: a.s.r (was Re: habeas - tainted white list)

Posted by Charles Gregory <cg...@hwcn.org>.
On Mon, 21 Dec 2009, J.D. Falk wrote:
> That's IT! PORNOGRAPHIC DOCUMENTATION!

Sorry. Already been tried.

But no matter what we called it, the users still didn't appreciate their 
computers or network going down on them. :)

- C
PS. Let's not get started on how hard disks are smaller than flippy ones...


a.s.r (was Re: habeas - tainted white list)

Posted by "J.D. Falk" <jd...@cybernothing.org>.
On Dec 18, 2009, at 11:34 AM, LuKreme wrote:

>> "Bother," said Pooh as he struggled with /etc/sendmail.cf, "it never
>> does quite what I want. I wish Christopher Robin was here."
>>                          -- Peter da Silva in a.s.r
> 
> That is truly brilliant. Not familiar with a.s.r though. Peter da Silva sounds familiar though.

Pete Krawczyk wrote :
> *sigh* Oh, how I wish lusers could read documentation more than they read
> porn...
That's IT! PORNOGRAPHIC DOCUMENTATION!

"...and as she finally reached orgasm, she screamed 'the mail server will be down for three hours tonight! Yes! Oh, yes!'"

http://home.xnet.com/~raven/Sysadmin/ASR.Quotes.html

(Sometimes I miss those days...the rest of the time I'm thankful I'm not a sysadmin anymore.)

--
J.D. Falk


Re: habeas - tainted white list

Posted by LuKreme <kr...@kreme.com>.
On Dec 18, 2009, at 7:12, John Hardin <jh...@impsec.org> wrote:
> On Fri, 18 Dec 2009, Christian Brel wrote:
>> On Fri, 18 Dec 2009 02:29:56 -0700
>> LuKreme <kr...@kreme.com> wrote:
>>> I might agree with some small portion of our resident troll's posts,
>>
>> You need to resort to abuse for what particular reason?
>
> Repeatedly accusing the SA developers of fraudulent collusion is  
> abusive. Don't be surprised if people are abusive in return.

I dunno. I don't consider Troll to be abusive. Descriptive, perhaps.


>  "Bother," said Pooh as he struggled with /etc/sendmail.cf, "it never
>  does quite what I want. I wish Christopher Robin was here."
>                           -- Peter da Silva in a.s.r

That is truly brilliant. Not familiar with a.s.r though. Peter da  
Silva sounds familiar though. 

Re: habeas - tainted white list

Posted by John Hardin <jh...@impsec.org>.
On Fri, 18 Dec 2009, Christian Brel wrote:

> On Fri, 18 Dec 2009 02:29:56 -0700
> LuKreme <kr...@kreme.com> wrote:
>
>> I might agree with some small portion of our resident troll's posts,
>
> You need to resort to abuse for what particular reason?

Repeatedly accusing the SA developers of fraudulent collusion is abusive. 
Don't be surprised if people are abusive in return.

-- 
  John Hardin KA7OHZ                    http://www.impsec.org/~jhardin/
  jhardin@impsec.org    FALaholic #11174     pgpk -a jhardin@impsec.org
  key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
   "Bother," said Pooh as he struggled with /etc/sendmail.cf, "it never
   does quite what I want. I wish Christopher Robin was here."
 				           -- Peter da Silva in a.s.r
-----------------------------------------------------------------------
  7 days until Christmas

Re: habeas - tainted white list

Posted by jdow <jd...@earthlink.net>.
From: "John Hardin" <jh...@impsec.org>
Sent: Friday, 2009/December/18 06:12


> On Fri, 18 Dec 2009, Christian Brel wrote:
>
>> On Fri, 18 Dec 2009 02:29:56 -0700
>> LuKreme <kr...@kreme.com> wrote:
>>
>>> I might agree with some small portion of our resident troll's posts,
>>
>> You need to resort to abuse for what particular reason?
>
> Repeatedly accusing the SA developers of fraudulent collusion is abusive. 
> Don't be surprised if people are abusive in return.

He customizes one element of his installation to quite thoroughly
"pass a lot of spam" settings. Then he whines when something HE
calls spam gets through. He expects Return Path and emailreg.org
to read his mind. And he refuses to make the simple corrections at
his end that would solve it for him and leave the rest of the world
properly protected. (He is NOT properly protected with his score
configuration.)

Just off hand I think this describes his bona fides to utterly ignore.

I wonder if a variant build of Spam Assassin could tag messages
coming through the list with an X-ChristianBrel header. On the Wiki
it'd be explained as "Meaningless noise from a fugghead." (That's
a willfully self-destructive person.)

Of course, /dev/null works. At least I don't see HIS messages. And I
could simply /dev/null the topic. Morbid curiosity keeps me watching
the thread.

{^_^} 


Re: habeas - tainted white list

Posted by Christian Brel <br...@copperproductions.co.uk>.
On Fri, 18 Dec 2009 02:29:56 -0700
LuKreme <kr...@kreme.com> wrote:

> I might agree with some small portion of our resident troll's posts,  

You need to resort to abuse for what particular reason?

-- 
This e-mail and any attachments may form pure opinion and may not have
any factual foundation. Please check any details provided to satisfy
yourself as to suitability or accuracy of any information provided.
Data Protection: Unless otherwise requested we may pass the information
you have provided to other partner organisations. 

Re: habeas - tainted white list

Posted by LuKreme <kr...@kreme.com>.
On Dec 18, 2009, at 2:07, "Daryl C. W. O'Shea"  
<sp...@dostech.ca> wrote:

> I stand firm on my opinion that our principle of safe for most users  
> is
> the logical reason for including DNSWLs.

Just to be clear, despite my dislike of the HABEAS rules, I am not a  
tinfoil-hat nutter thinking there's some conspiracy. I even had quite  
good result with HABEAS way back when. My problems were purely a  
result of getting occasional waves of miss-classed spam getting  
through because of HABEAS.

I might agree with some small portion of our resident troll's posts,  
but I am still a big fan of SA and an eagerly awaiting the release of  
3.3.
  
   

Re: habeas - tainted white list

Posted by "Daryl C. W. O'Shea" <sp...@dostech.ca>.
On 18/12/2009 3:32 AM, Christian Brel wrote:
> On Fri, 18 Dec 2009 02:24:45 -0500
> "Daryl C. W. O'Shea" <sp...@dostech.ca> wrote:
> 
>> Reputation type rules (such as DNSWLs) are probably the only (or
>> certainly one of the very few) types of rules that you can weight
>> heavily negatively.  This is due to the nature of an open source
>> product (or even given enough time to game a closed source product).
>> Content based rules are very often easily beaten.  If we could have a
>> body rule that looks for "this mail is good" and assign a -20 score
>> we would. Clearly that would not work.
> 
> With the kindest of respect, I have to disagree with this.

How the following text supports your disagreement I don't know.  But
I'll agree to disagree.

> If for
> argument sake five blocklists with no business {or other} relationship
> with Spamassassin  flag an IP for spamming, then it's a good bet
> that they are correct and any perceived negativity is earned. How this
> impacts on Spamassassin is dependent on the scores set - which comes
> back to you and the developers - so the arguement not only has not
> legs, it has no arms either. Consider that blocklists are often
> universally trusted to be sat on the SMTP connection level ahead of
> Spamassassin, whereas the suggestion of doing that with Habeas as a
> whitelist would be pure comedy gold :-)
> 
>> Again, find me a commercial white list that wants to be included in
>> SpamAssassin on a "free for use basis" and I'll pay for the phone call
>> to talk to them.  Seriously.
> I shake my head in utter disbelief at this comment, and I'm sure that
> Apache Sponsor Barracuda AKA 'emailreg.org' will have just pricked up
> their ears. 

So what if they do.  We'll test it and judge it on stats (not random FPs
or stories about friends who had a bad employment experience).  If it
works good it works good, if it doesn't we won't use it and they'll
understand.

>> I'm pretty sure I brought up the SA developers' *long* standing
>> principle of being as safe as possible for the majority of users by
>> erring on the side of missing spam rather than tagging ham while still
>> putting out a useful product.
> 
> It's a fair statement that in using an Antispam 'product' that blocks
> nothing and only assigns a score, the issue of having that score
> reduced in favour of a known commercial bulk mailer is undesirable.

Just so I'm clear, are you equating all commercial bulk mail to spam?  I
would disagree if that is the case.  You would likely disagree with me
and then I would agree to disagree.

> The statistics may have some interest but can be applied to show there
> is little cause to keep the rule at all if you so wish to bend it the
> other way.

I've already explained my rationale for keeping it.  It's a small trade
off to cover the unknown.  Our ham corpus is not that large.

> The key is this: I would *never* have known what HABEAS was
> if I had not seen the name in low scoring spam and asked why. It does
> not look like I'm the first to ask either.

You know, it's funny you mention it.  I've found out about some
blacklists, even ones now included in SpamAssassin, only because they
caught one-to-one personal emails (that no-one could argue were
commercial) of random people that I know (and who have inquired about
the block).

>> From the data we have from mass-checks we are erring a very small
>> amount on the side of caution by not disabling the whitelists by
>> default.
> It's a big fat favourable score to one organisation for 'erring a very
> small amount on the side of caution' don't you think? -4/-8 given the
> average 419 spam only scores 4-8 points.

Again, we agree.  We've changed it in the upcomming release and will
surely backport it when we're done getting 3.3 out.  It's been like this
for years, I don't think we need to jump like crazy to change the 3.2
updates before we've even settled on a final score.

> Forgive me but are Return Path
> pulling someones strings here as Puppet Masters?

I really wish they would.  I sure could use the money.  In 6 or so years
of SA development I've netted me a total of... a $30 book (Thanks Dan!).
 If I were to sell that book I'd be a small way towards covering this
month's costs for the sa-update mirrors I run out of my own pocket.

> If everything is open and transparent give the default user the option
> to *enable* them and score them zero, unless - of course - there is
> some kind of logical reason for these mad scoring spam assisting rules
> that favour Return Path in the default set up?

I stand firm on my opinion that our principle of safe for most users is
the logical reason for including DNSWLs.

If you like you can transparently disable the DNSWLs.

Daryl


Re: habeas - tainted white list

Posted by Christian Brel <br...@copperproductions.co.uk>.
On Fri, 18 Dec 2009 02:21:00 -0700
LuKreme <kr...@kreme.com> wrote:

> On Dec 18, 2009, at 1:32, Christian Brel
> <brel.spamassassin091214@copperproductions.co.uk 
>  > wrote:
> 
> > the issue of having that score
> > reduced in favour of a known commercial bulk mailer is undesirable.
> 
> The trouble is you seem to consider ALL commercial senders to be  
> spammers. That's just not true.
>   
No, I don't. But I do consider many commercial emailers to abuse
personal data for their own gain. To me it is spam if it does not
directly relate to a transaction that I have instigated. If it's
special offers, news or other marketing rubbish aimed at selling me
something or telling me about new services - it's spam.

We've moved on since the Tandy/Radio Shack days of data collected at
the point of sale forever being used to abuse you forever more.

-- 
This e-mail and any attachments may form pure opinion and may not have
any factual foundation. Please check any details provided to satisfy
yourself as to suitability or accuracy of any information provided.
Data Protection: Unless otherwise requested we may pass the information
you have provided to other partner organisations. 

Re: habeas - tainted white list

Posted by LuKreme <kr...@kreme.com>.
On Dec 18, 2009, at 1:32, Christian Brel <brel.spamassassin091214@copperproductions.co.uk 
 > wrote:

> the issue of having that score
> reduced in favour of a known commercial bulk mailer is undesirable.

The trouble is you seem to consider ALL commercial senders to be  
spammers. That's just not true.
  

Re: habeas - tainted white list

Posted by Christian Brel <br...@copperproductions.co.uk>.
On Fri, 18 Dec 2009 02:24:45 -0500
"Daryl C. W. O'Shea" <sp...@dostech.ca> wrote:

> Reputation type rules (such as DNSWLs) are probably the only (or
> certainly one of the very few) types of rules that you can weight
> heavily negatively.  This is due to the nature of an open source
> product (or even given enough time to game a closed source product).
> Content based rules are very often easily beaten.  If we could have a
> body rule that looks for "this mail is good" and assign a -20 score
> we would. Clearly that would not work.

With the kindest of respect, I have to disagree with this. If for
argument sake five blocklists with no business {or other} relationship
with Spamassassin  flag an IP for spamming, then it's a good bet
that they are correct and any perceived negativity is earned. How this
impacts on Spamassassin is dependent on the scores set - which comes
back to you and the developers - so the arguement not only has not
legs, it has no arms either. Consider that blocklists are often
universally trusted to be sat on the SMTP connection level ahead of
Spamassassin, whereas the suggestion of doing that with Habeas as a
whitelist would be pure comedy gold :-)

> Again, find me a commercial white list that wants to be included in
> SpamAssassin on a "free for use basis" and I'll pay for the phone call
> to talk to them.  Seriously.
I shake my head in utter disbelief at this comment, and I'm sure that
Apache Sponsor Barracuda AKA 'emailreg.org' will have just pricked up
their ears. 

> I'm pretty sure I brought up the SA developers' *long* standing
> principle of being as safe as possible for the majority of users by
> erring on the side of missing spam rather than tagging ham while still
> putting out a useful product.

It's a fair statement that in using an Antispam 'product' that blocks
nothing and only assigns a score, the issue of having that score
reduced in favour of a known commercial bulk mailer is undesirable.
The statistics may have some interest but can be applied to show there
is little cause to keep the rule at all if you so wish to bend it the
other way. The key is this: I would *never* have known what HABEAS was
if I had not seen the name in low scoring spam and asked why. It does
not look like I'm the first to ask either.

> 
> From the data we have from mass-checks we are erring a very small
> amount on the side of caution by not disabling the whitelists by
> default.
It's a big fat favourable score to one organisation for 'erring a very
small amount on the side of caution' don't you think? -4/-8 given the
average 419 spam only scores 4-8 points. Forgive me but are Return Path
pulling someones strings here as Puppet Masters?

If everything is open and transparent give the default user the option
to *enable* them and score them zero, unless - of course - there is
some kind of logical reason for these mad scoring spam assisting rules
that favour Return Path in the default set up?



-- 
This e-mail and any attachments may form pure opinion and may not have
any factual foundation. Please check any details provided to satisfy
yourself as to suitability or accuracy of any information provided.
Data Protection: Unless otherwise requested we may pass the information
you have provided to other partner organisations. 

Re: habeas - tainted white list

Posted by jdow <jd...@earthlink.net>.
From: "John Hardin" <jh...@impsec.org>
Sent: Friday, 2009/December/18 08:07


> On Fri, 18 Dec 2009, Christian Brel wrote:
>
>> On Fri, 18 Dec 2009 06:19:25 -0800 (PST)
>> John Hardin <jh...@impsec.org> wrote:
>>
>>> We understand your philosophical objection. Providing hard evidence
>>> of FNs will go much further towards making your point than name
>>> calling will.
>>
>> The name calling being?
>
> Alright, let me amend that: Providing hard evidence of FNs will go much 
> further towards making your point - and getting the rules fixed in a 
> useful manner - than will repeated accusations that the SA devs are taking 
> bribes to weaken SA.
>
> And phrasing it as a question doesn't make it any less of an accusation, 
> given it keeps being repeated after reasonable explanations have been 
> provided.
>
> At the moment there's insufficient _hard data_ to support the contention 
> that the reputation whitelists are assisting FNs to a great degree. The 
> data from masscheck suggests the impact of the reputation whitelists is 
> neutral to very slightly positive (in terms of reducing FPs). If you feel 
> this isn't justified, if you're seeing a lot of FNs that can be laid at 
> the feet of a reputation whitelist rule, then please feed that hard data 
> into the masscheck corpora so that the scoring process can take it into 
> account.

John, he is a teleological thinker. Epistemological arguments do not
mean a thing to him. Reality is consensual to him. He thinks he can
bend reality to his will and all spam will go away because he forced
somebody else to cripple a product.

Forget it, teleological thinkers are impervious to logic. Ignore the
twit.

{^_^} 


Re: habeas - tainted white list

Posted by Christian Brel <br...@copperproductions.co.uk>.
On Fri, 18 Dec 2009 06:19:25 -0800 (PST)
John Hardin <jh...@impsec.org> wrote:

> On Fri, 18 Dec 2009, Christian Brel wrote:
> 
> > On Fri, 18 Dec 2009 06:49:41 -0600
> > Daniel J McDonald <da...@austinenergy.com> wrote:
> >
> >> On Fri, 2009-12-18 at 08:49 +0000, Christian Brel wrote:
> >>> On Fri, 18 Dec 2009 03:44:32 -0500
> >>> "Daryl C. W. O'Shea" <sp...@dostech.ca> wrote:
> >>>
> >>>> Please stop beating the -4 and -8 horse.  We agree.
> >>>
> >>> Then fix it and show who really is in charge of this project?
> >>
> >> It's been fixed.  Don't you know how to use bugzilla?
> >>
> >> http://svn.apache.org/viewvc/spamassassin/trunk/rules/50_scores.cf?r1=891460&r2=891459&pathrev=891460
> >>
> >> The new scores will come out in 3.3.0, RC1 is very soon...
> >
> > +score RCVD_IN_RP_CERTIFIED 0.0 -3.0 0.0 -3.0
> > +score RCVD_IN_RP_SAFE 0.0 -2.0 0.0 -2.0
> >
> > This is 'fixed'?
> 
> In the absence of evidence to the contrary, yes.
> 
> If it's that big a problem for you in real life, then you should be
> able to provide FNs to the masscheck corpora that will _prove_ these
> scores are too generous.
> 
> We understand your philosophical objection. Providing hard evidence
> of FNs will go much further towards making your point than name
> calling will.
> 
The name calling being?


-- 
This e-mail and any attachments may form pure opinion and may not have
any factual foundation. Please check any details provided to satisfy
yourself as to suitability or accuracy of any information provided.
Data Protection: Unless otherwise requested we may pass the information
you have provided to other partner organisations. 

Re: habeas - tainted white list

Posted by "Daryl C. W. O'Shea" <sp...@dostech.ca>.
On 18/12/2009 2:44 PM, Rob McEwen wrote:
> R-Elists wrote:
>> here is a chance for possible help in more areas than just this specific
>> ruleset issue...
>>
>> i asked Rob some time ago if he could write a script that would check logs
>> and report if a certain rule was effective or not by itself vrs if other
>> rules hit with it and maybe that rule was not needed or could be lowered etc

Well it doesn't report to alert people that a rule may not make much of
a difference in the scheme of things, you can infer the information from
ruleqa's score map output.

Daryl


Re: habeas - tainted white list

Posted by Rob McEwen <ro...@invaluement.com>.
R-Elists wrote:
> here is a chance for possible help in more areas than just this specific
> ruleset issue...
>
> i asked Rob some time ago if he could write a script that would check logs
> and report if a certain rule was effective or not by itself vrs if other
> rules hit with it and maybe that rule was not needed or could be lowered etc
> etc
>
> and if other rules hit with it, then we would see how effective that rule
> was and why and when etc etc
>
> i am guessing that you folks already have these tools or similar tools or
> help?
>   

This is still on my "to do" list, but duties with invaluement.com only
keep growing, so it is hard to prioritize this. But I find it hard to
believe that this doesn't already exist. All that is needed is a plug-in
that would copy to a specified directory all messages which hit on X
rule (and/or dnsbl). The plug-in would be able to (optionally) only take
action if the message scored either "at or above threshold" or "below
threshold". Then, whenever testing a new rule/dnsbl, simply score it at
0.01, point the plugin at that rule or dnsbl, and have it only act on
messages which scored "below threshold".

This would be extremely valuable for determining the following about a
new rule or DNSBL:

(1) How much spam the rule would have blocked if being used aggressively
(but was missed with the 0.01 score) and, therefore, made it to the
inbox during the testing phase because nothing else in production had
stopped it?

(2) How many legit messages would have been blocked with the use of this
rule or DNSBL? (FPs)

Of course, BOTH of those examples would consist of messages which scored
"below threshold" even while hitting on that new rule (given its 0.01
score). So it would be up to the e-mail administrator to then examine
the messages and judge for themselves whether these were FPs, or
would-have-missed-without-the-new-rule spams (aka corrected FNs).

If anyone ever develops such a plugin before I have time to, PLEASE let
me know!

-- 
Rob McEwen
http://dnsbl.invaluement.com/
rob@invaluement.com
+1 (478) 475-9032



RE: habeas - tainted white list

Posted by R-Elists <li...@abbacomm.net>.
> 
> In the absence of evidence to the contrary, yes.
> 
> If it's that big a problem for you in real life, then you 
> should be able to provide FNs to the masscheck corpora that 
> will _prove_ these scores are too generous.
> 
> We understand your philosophical objection. Providing hard 
> evidence of FNs will go much further towards making your 
> point than name calling will.
> 
> -- 
>   John Hardin 

John,

great!!!

here is a chance for possible help in more areas than just this specific
ruleset issue...

i asked Rob some time ago if he could write a script that would check logs
and report if a certain rule was effective or not by itself vrs if other
rules hit with it and maybe that rule was not needed or could be lowered etc
etc

and if other rules hit with it, then we would see how effective that rule
was and why and when etc etc

i am guessing that you folks already have these tools or similar tools or
help?

although i could probably come up with general logic flow and an algo for
this, i would not be able to hard codify and implement at this time...

yeah yeah, i know and im still working with PERL for dummies and will get
past the intro some time soon

 - rh


Re: habeas - tainted white list

Posted by John Hardin <jh...@impsec.org>.
On Fri, 18 Dec 2009, Christian Brel wrote:

> On Fri, 18 Dec 2009 06:49:41 -0600
> Daniel J McDonald <da...@austinenergy.com> wrote:
>
>> On Fri, 2009-12-18 at 08:49 +0000, Christian Brel wrote:
>>> On Fri, 18 Dec 2009 03:44:32 -0500
>>> "Daryl C. W. O'Shea" <sp...@dostech.ca> wrote:
>>>
>>>> Please stop beating the -4 and -8 horse.  We agree.
>>>
>>> Then fix it and show who really is in charge of this project?
>>
>> It's been fixed.  Don't you know how to use bugzilla?
>>
>> http://svn.apache.org/viewvc/spamassassin/trunk/rules/50_scores.cf?r1=891460&r2=891459&pathrev=891460
>>
>> The new scores will come out in 3.3.0, RC1 is very soon...
>
> +score RCVD_IN_RP_CERTIFIED 0.0 -3.0 0.0 -3.0
> +score RCVD_IN_RP_SAFE 0.0 -2.0 0.0 -2.0
>
> This is 'fixed'?

In the absence of evidence to the contrary, yes.

If it's that big a problem for you in real life, then you should be able 
to provide FNs to the masscheck corpora that will _prove_ these scores are 
too generous.

We understand your philosophical objection. Providing hard evidence of FNs 
will go much further towards making your point than name calling will.

-- 
  John Hardin KA7OHZ                    http://www.impsec.org/~jhardin/
  jhardin@impsec.org    FALaholic #11174     pgpk -a jhardin@impsec.org
  key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
   "Bother," said Pooh as he struggled with /etc/sendmail.cf, "it never
   does quite what I want. I wish Christopher Robin was here."
 				           -- Peter da Silva in a.s.r
-----------------------------------------------------------------------
  7 days until Christmas

Re: habeas - tainted white list

Posted by Daniel J McDonald <da...@austinenergy.com>.
On Fri, 2009-12-18 at 12:53 +0000, Christian Brel wrote:
> On Fri, 18 Dec 2009 06:49:41 -0600
> Daniel J McDonald <da...@austinenergy.com> wrote:
> 
> > On Fri, 2009-12-18 at 08:49 +0000, Christian Brel wrote:
> > > On Fri, 18 Dec 2009 03:44:32 -0500
> > > "Daryl C. W. O'Shea" <sp...@dostech.ca> wrote:
> > > 
> > > > Please stop beating the -4 and -8 horse.  We agree.
> > > > 
> > > > Daryl
> > > > 
> > > > 
> > > 
> > > Then fix it and show who really is in charge of this project?
> > > 
> > It's been fixed.  Don't you know how to use bugzilla?
> > 
> > http://svn.apache.org/viewvc/spamassassin/trunk/rules/50_scores.cf?r1=891460&r2=891459&pathrev=891460
> > 
> > The new scores will come out in 3.3.0, RC1 is very soon...
> > 
> 
> +score RCVD_IN_RP_CERTIFIED 0.0 -3.0 0.0 -3.0
> +score RCVD_IN_RP_SAFE 0.0 -2.0 0.0 -2.0
> 
> This is 'fixed'? 

Have you read the bugzilla entry?  huge discussion about how to fix it
properly.  You also ignored the five rules removed and replaced by these
two.


-- 
Daniel J McDonald, CCIE # 2495, CISSP # 78281, CNX
www.austinenergy.com

Re: habeas - tainted white list

Posted by Christian Brel <br...@copperproductions.co.uk>.
On Fri, 18 Dec 2009 06:49:41 -0600
Daniel J McDonald <da...@austinenergy.com> wrote:

> On Fri, 2009-12-18 at 08:49 +0000, Christian Brel wrote:
> > On Fri, 18 Dec 2009 03:44:32 -0500
> > "Daryl C. W. O'Shea" <sp...@dostech.ca> wrote:
> > 
> > > Please stop beating the -4 and -8 horse.  We agree.
> > > 
> > > Daryl
> > > 
> > > 
> > 
> > Then fix it and show who really is in charge of this project?
> > 
> It's been fixed.  Don't you know how to use bugzilla?
> 
> http://svn.apache.org/viewvc/spamassassin/trunk/rules/50_scores.cf?r1=891460&r2=891459&pathrev=891460
> 
> The new scores will come out in 3.3.0, RC1 is very soon...
> 

+score RCVD_IN_RP_CERTIFIED 0.0 -3.0 0.0 -3.0
+score RCVD_IN_RP_SAFE 0.0 -2.0 0.0 -2.0

This is 'fixed'? 

-- 
This e-mail and any attachments may form pure opinion and may not have
any factual foundation. Please check any details provided to satisfy
yourself as to suitability or accuracy of any information provided.
Data Protection: Unless otherwise requested we may pass the information
you have provided to other partner organisations. 

Re: habeas - tainted white list

Posted by Daniel J McDonald <da...@austinenergy.com>.
On Fri, 2009-12-18 at 08:49 +0000, Christian Brel wrote:
> On Fri, 18 Dec 2009 03:44:32 -0500
> "Daryl C. W. O'Shea" <sp...@dostech.ca> wrote:
> 
> > Please stop beating the -4 and -8 horse.  We agree.
> > 
> > Daryl
> > 
> > 
> 
> Then fix it and show who really is in charge of this project?
> 
It's been fixed.  Don't you know how to use bugzilla?

http://svn.apache.org/viewvc/spamassassin/trunk/rules/50_scores.cf?r1=891460&r2=891459&pathrev=891460

The new scores will come out in 3.3.0, RC1 is very soon...

-- 
Daniel J McDonald, CCIE # 2495, CISSP # 78281, CNX
www.austinenergy.com

Re: habeas - tainted white list

Posted by Christian Brel <br...@copperproductions.co.uk>.
On Fri, 18 Dec 2009 03:44:32 -0500
"Daryl C. W. O'Shea" <sp...@dostech.ca> wrote:

> Please stop beating the -4 and -8 horse.  We agree.
> 
> Daryl
> 
> 

Then fix it and show who really is in charge of this project?

-- 
This e-mail and any attachments may form pure opinion and may not have
any factual foundation. Please check any details provided to satisfy
yourself as to suitability or accuracy of any information provided.
Data Protection: Unless otherwise requested we may pass the information
you have provided to other partner organisations. 

Re: habeas - tainted white list

Posted by "Daryl C. W. O'Shea" <sp...@dostech.ca>.
On 18/12/2009 3:09 AM, LuKreme wrote:
> On 18-Dec-2009, at 00:24, Daryl C. W. O'Shea wrote:
>> From the data we have from mass-checks we are erring a very small amount
>> on the side of caution by not disabling the whitelists by default.
> 
> 
> I guess that the real issue that I have with the whole HABEAS thing is the magnitude of the default scores. −4 and −8 caused issues that would never have arisen had the defaults been −0.4 and −0.8. Or even −1 and −2.

The scores have been decreased in the upcoming proposed release ruleset.
 Not to -0.4 and -0.8, but they're no longer -4 and -8.  I'm sure that
we'll get to (it's been -4 and -8 for years, we're not in a huge rush to
do anything now) decreasing them in the 3.2.x sa-update ruleset also
once we've firmed up an opinion of what they should be going forward.

Please stop beating the -4 and -8 horse.  We agree.

Daryl



Re: habeas - tainted white list

Posted by LuKreme <kr...@kreme.com>.
On 18-Dec-2009, at 00:24, Daryl C. W. O'Shea wrote:
> From the data we have from mass-checks we are erring a very small amount
> on the side of caution by not disabling the whitelists by default.


I guess that the real issue that I have with the whole HABEAS thing is the magnitude of the default scores. −4 and −8 caused issues that would never have arisen had the defaults been −0.4 and −0.8. Or even −1 and −2.

-- 
The fact is that camels are far more intelligent than dolphins. Footnote: Never trust a species that grins all the time. It's up to something. --Pyramids


Re: habeas - tainted white list

Posted by "Daryl C. W. O'Shea" <sp...@dostech.ca>.
On 18/12/2009 8:35 AM, Per Jessen wrote:
> Daryl C. W. O'Shea wrote:
> 
>> If we had more mass-check data from a wider number of mail recipients
>> maybe it would change things, statistically, maybe it wouldn't.  New
>> mass-check contributors are always welcome.  They take very little
>> effort to manage once you've set it up (I ignore mine for years at a
>> time).
> 
> Is there a good howto for setting this up? 

Other than a clean corpus, it doesn't take much more effort:

http://wiki.apache.org/spamassassin/NightlyMassCheck

Daryl


Re: habeas - tainted white list

Posted by Per Jessen <pe...@computer.org>.
Daryl C. W. O'Shea wrote:

> If we had more mass-check data from a wider number of mail recipients
> maybe it would change things, statistically, maybe it wouldn't.  New
> mass-check contributors are always welcome.  They take very little
> effort to manage once you've set it up (I ignore mine for years at a
> time).

Is there a good howto for setting this up? 


/Per Jessen, Zürich


Re: habeas - tainted white list

Posted by "Daryl C. W. O'Shea" <sp...@dostech.ca>.
On 18/12/2009 1:11 AM, Christian Brel wrote:
> On Thu, 17 Dec 2009 15:51:35 -0500
> "Daryl C. W. O'Shea" <sp...@dostech.ca> wrote:
> 
> 
>> I think the current score changes are a good step.  Another step may
>> be including in the release notes that there are whitelists and that
>> people may want to disable them by score whatever rules (a list of
>> them) 0.
> 
> Why not default them to zero and include in the release notes/man that
> there are whitelists and they can *enable* them?

I'm pretty sure I brought up the SA developers' *long* standing
principle of being as safe as possible for the majority of users by
erring on the side of missing spam rather than tagging ham while still
putting out a useful product.

>From the data we have from mass-checks we are erring a very small amount
on the side of caution by not disabling the whitelists by default.

If we had more mass-check data from a wider number of mail recipients
maybe it would change things, statistically, maybe it wouldn't.  New
mass-check contributors are always welcome.  They take very little
effort to manage once you've set it up (I ignore mine for years at a time).

Daryl




Re: habeas - tainted white list

Posted by jdow <jd...@earthlink.net>.
From: "Christian Brel" <br...@copperproductions.co.uk>
Sent: Thursday, 2009/December/17 22:11


> On Thu, 17 Dec 2009 15:51:35 -0500
> "Daryl C. W. O'Shea" <sp...@dostech.ca> wrote:
> 
> 
>> I think the current score changes are a good step.  Another step may
>> be including in the release notes that there are whitelists and that
>> people may want to disable them by score whatever rules (a list of
>> them) 0.
> 
> Why not default them to zero and include in the release notes/man that
> there are whitelists and they can *enable* them?

Because we enjoy tweaking the nose of idiots?

{O,o}    <- being wonked out silly, which is all you deserve.

Re: habeas - tainted white list

Posted by Christian Brel <br...@copperproductions.co.uk>.
On Thu, 17 Dec 2009 15:51:35 -0500
"Daryl C. W. O'Shea" <sp...@dostech.ca> wrote:


> I think the current score changes are a good step.  Another step may
> be including in the release notes that there are whitelists and that
> people may want to disable them by score whatever rules (a list of
> them) 0.

Why not default them to zero and include in the release notes/man that
there are whitelists and they can *enable* them?
> 

-- 
This e-mail and any attachments may form pure opinion and may not have
any factual foundation. Please check any details provided to satisfy
yourself as to suitability or accuracy of any information provided.
Data Protection: Unless otherwise requested we may pass the information
you have provided to other partner organisations. 

Re: habeas - tainted white list

Posted by "Daryl C. W. O'Shea" <sp...@dostech.ca>.
On 17/12/2009 2:21 PM, R-Elists wrote:
> ...based upon Togami's data processing, the biggest thing that comes to mind
> is this...
> 
> *IF* these or similar rulesets are not truly not making a difference one way
> or the other, then why are they there?
> 
> why do we really need them or the other similar rulesets?

We can't and aren't really sure that they don't make a difference.  Our
ham corpus isn't really all that big.  For the most part it's probably
made up largely of types of mail that Return-Path wouldn't be dealing
with on their lists.  Clearly it's not containing much mail that
Return-Path deals with.  The corpus isn't big enough to say that most
people (and most people aren't technical people, rather are just common
Internet users) won't get mail that Return-Path doesn't deal with though.

> ...and why should any rules "such as these" have a default SA installation
> value other than "zero" and then educate admins in the documentation what to
> do in regards to enabling and suggested scoring?

SA is designed to be safe for most users.  Most as in general Internet
users and safe as in it would rather not tag mail than tag it.

IMO whitelists have a place in SA, even whitelists that we cannot
determine due to a small corpus size whether or not they're actually
making a difference... at least when based on our corpus there's no
evidence that they're statistically and drastically causing a
significant amount of spam to pass that otherwise wouldn't.

We treat blacklists the same way.  We include blacklists in the default
install to stop spam.  We include whitelists because of our core
principle of being safe for most users in general.

I think the current score changes are a good step.  Another step may be
including in the release notes that there are whitelists and that people
may want to disable them by score whatever rules (a list of them) 0.

BTW, I will not waste any cycles defending individual instances on spam
getting by because of whitelists for the exact same reason that I do not
do the same for ham that gets caught by whitelists.

Daryl


RE: [sa] Re: habeas - tainted white list

Posted by R-Elists <li...@abbacomm.net>.
> I believe on the whole Warren Togami's posting about a 
> whitelist performance on a masscheck settles the affair. 
> White lists are very reliable. They are also very unnecessary 
> within SpamAssassin. So perhaps the whole topic can die.
> 
> I also note that the people complaining about the white lists 
> seem to leave out solid data. Were the "spams" really 
> confirmed spams or were they merely scored as spams? What 
> scores hit that made them score as spams? What kind of 
> installation do you have? How many emails a day are processed?
> 
> It's little details like that which prompt other people to 
> look at assertions somewhat askance or ignore them outright.
> 
> With my three personal accounts I have yet to see an email 
> off this list containing HABEAS, spam or ham, since this 
> discussion began. I guess I don't do business with HABEAS 
> customers and no spammers have pushed through anything from a 
> HABEAS site. The mail volume is fairly high (LKML and a 
> couple other Linux lists). And the spam seems to be suddenly 
> up from 60-80 a day to the 90s/day. For those spammers who 
> are listening, I REALLY do not need Via-thingie-alis whether 
> or not it is from he Pf people. If I REALLY need to get it up 
> I do a sexy striptease or something like that. (The V thingie 
> seems to be a new feature of my spam bucket - 10 or more of 
> them a day.)
> 
> {^_-}
> 

JDow et al,

why do you say "on the whole" ? what is holding you back in your thinking
there?

...based upon Togami's data processing, the biggest thing that comes to mind
is this...

*IF* these or similar rulesets are not truly not making a difference one way
or the other, then why are they there?

why do we really need them or the other similar rulesets?

...and why should any rules "such as these" have a default SA installation
value other than "zero" and then educate admins in the documentation what to
do in regards to enabling and suggested scoring?

 - rh


Re: [sa] Re: habeas - tainted white list

Posted by jdow <jd...@earthlink.net>.
From: "Christian Brel" <br...@copperproductions.co.uk>
Sent: Thursday, 2009/December/17 09:28


> {side note}
> Has anyone noticed how the thread 'emailreg.org - tainted white list'
> has been left unchanged, despite the topic moving on to Habeas. Whilst
> this is side splittingly funny if you do a search on emailreg.org and
> see it in the archives, it's probably not fair to drag their name
> through the mud when the topic has moved on?
> 
> I wonder how long the thread will be left at the new 're: habeas -
> tainted white list'? How many will post using it? Or if those black
> helicopters and MIB's will seek to put a stop to it?

I believe on the whole Warren Togami's posting about a whitelist
performance on a masscheck settles the affair. White lists are very
reliable. They are also very unnecessary within SpamAssassin. So
perhaps the whole topic can die.

I also note that the people complaining about the white lists seem
to leave out solid data. Were the "spams" really confirmed spams or
were they merely scored as spams? What scores hit that made them
score as spams? What kind of installation do you have? How many
emails a day are processed?

It's little details like that which prompt other people to look at
assertions somewhat askance or ignore them outright.

With my three personal accounts I have yet to see an email off this
list containing HABEAS, spam or ham, since this discussion began. I
guess I don't do business with HABEAS customers and no spammers have
pushed through anything from a HABEAS site. The mail volume is fairly
high (LKML and a couple other Linux lists). And the spam seems to be
suddenly up from 60-80 a day to the 90s/day. For those spammers who
are listening, I REALLY do not need Via-thingie-alis whether or not
it is from he Pf people. If I REALLY need to get it up I do a sexy
striptease or something like that. (The V thingie seems to be a new
feature of my spam bucket - 10 or more of them a day.)

{^_-}

Re: [sa] Re: habeas - tainted white list

Posted by Christian Brel <br...@copperproductions.co.uk>.
{side note}
Has anyone noticed how the thread 'emailreg.org - tainted white list'
has been left unchanged, despite the topic moving on to Habeas. Whilst
this is side splittingly funny if you do a search on emailreg.org and
see it in the archives, it's probably not fair to drag their name
through the mud when the topic has moved on?

I wonder how long the thread will be left at the new 're: habeas -
tainted white list'? How many will post using it? Or if those black
helicopters and MIB's will seek to put a stop to it?


-- 
This e-mail and any attachments may form pure opinion and may not have
any factual foundation. Please check any details provided to satisfy
yourself as to suitability or accuracy of any information provided.
Data Protection: Unless otherwise requested we may pass the information
you have provided to other partner organisations.