You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Reindl Harald <h....@thelounge.net> on 2014/09/16 12:24:01 UTC
LIST_PARTIAL
Hi
score LIST_PARTIAL 2.000 1.999 2.000 1.999
that feels too high, as example we add "List-Unsubscribe"
headers in case of ordiany newsletters to support MUA
which read that header (for TB a extension exists)
IMHO that penalty hits senders which try to be responsible
http://blog.sendblaster.com/2010/09/07/adding-list-unsubscribe-header-improves-reliability-and-reduce-complaints/
http://blog.mailchimp.com/gmails-new-unsubscribe-link-and-feedback-loop/
Re: LIST_PARTIAL
Posted by Reindl Harald <h....@thelounge.net>.
Am 16.09.2014 um 17:11 schrieb RW:
> On Tue, 16 Sep 2014 16:07:47 +0200
> Reindl Harald wrote:
>
>> in that case not, but depends on some wired HTML, bad chosen
>> subject or so hit another rule where 2 points penalty may
>> come closer to spam, keep in mind nobody knows how good or
>> bad a bayes on the RCPT side maybe trained
>>
>> what alarmed me was not that our SA in that specific case
>> had -1 and not -3 but that a message with a header added
>> by good intentions get a high penalty in general
>
> Very little of my own mail with unsubscribe headers would FP on
> LIST_PARTIAL because practically all of it has DKIM
>
> __LIST_PARTIAL && !__BUGGED_IMG && !__DKIM_EXISTS && !__RP_MATCHES_RCVD && !__HAS_SENDER
well, no DKIM here for now, too many domains and servers
to implement that without take care and time, but SPF_PASS
as said: i added the List-ID with "Newsletter <newsletter.domain.tld>"
to prevent get that score on other destinations using SA
BEFORE: -1 BAYES_00,CUST_DNSWL_8,HTML_MESSAGE,LIST_PARTIAL,SPF_PASS
NOW: -3 BAYES_00,CUST_DNSWL_8,HTML_MESSAGE,SPF_PASS
Re: LIST_PARTIAL
Posted by RW <rw...@googlemail.com>.
On Tue, 16 Sep 2014 16:07:47 +0200
Reindl Harald wrote:
> in that case not, but depends on some wired HTML, bad chosen
> subject or so hit another rule where 2 points penalty may
> come closer to spam, keep in mind nobody knows how good or
> bad a bayes on the RCPT side maybe trained
>
> what alarmed me was not that our SA in that specific case
> had -1 and not -3 but that a message with a header added
> by good intentions get a high penalty in general
Very little of my own mail with unsubscribe headers would FP on
LIST_PARTIAL because practically all of it has DKIM
__LIST_PARTIAL && !__BUGGED_IMG && !__DKIM_EXISTS && !__RP_MATCHES_RCVD && !__HAS_SENDER
Re: LIST_PARTIAL
Posted by Reindl Harald <h....@thelounge.net>.
Am 16.09.2014 um 16:20 schrieb John Hardin:
> On Tue, 16 Sep 2014, Reindl Harald wrote:
>> Am 16.09.2014 um 16:01 schrieb John Hardin:
>>> On Tue, 16 Sep 2014, Reindl Harald wrote:
>>>>> Le 16/09/2014 12:24, Reindl Harald a écrit :
>>>>>> score LIST_PARTIAL 2.000 1.999 2.000 1.999
>>>>>>
>>>>>> that feels too high, as example we add "List-Unsubscribe" headers in case of ordiany newsletters to support MUA
>>>>>> which read that header (for TB a extension exists)
>>>>
>>>> i have seen that rule hit one customers newsletter generated here, going through a relay responsible for the
>>>> sender domain and coming back to the customer itself why it makes it through SA
>>>
>>> Important question: are those messages being scored as spam?
>>>
>>> If not, how close to spam (5.0) are they being scored?
>>> Are you willing to provide a sample of one such?
>>
>> in that case not, but depends on some wired HTML, bad chosen
>> subject or so hit another rule where 2 points penalty may
>> come closer to spam, keep in mind nobody knows how good or
>> bad a bayes on the RCPT side maybe trained
>>
>> what alarmed me was not that our SA in that specific case
>> had -1 and not -3 but that a message with a header added
>> by good intentions get a high penalty in general
>
> 2 points is not a "high penalty". 4+ points is a "high penalty".
depends on the environment, in case of a wrong trained bayse (all sort of
newsletters trained as spam instead unsubscribe which is also the reason for
many RBL entries, frankly i saw persons report the newsletter sent from the
own brothers company to AOL as spam instead click on unsubscribe) it may be
the 2 points lead to reject the message :-)
> Regardless, I've reviewed the recent performance and adjusted the score limit a bit
thank you
Re: LIST_PARTIAL
Posted by John Hardin <jh...@impsec.org>.
On Tue, 16 Sep 2014, Reindl Harald wrote:
> Am 16.09.2014 um 16:01 schrieb John Hardin:
>> On Tue, 16 Sep 2014, Reindl Harald wrote:
>>>> Le 16/09/2014 12:24, Reindl Harald a écrit :
>>>>> score LIST_PARTIAL 2.000 1.999 2.000 1.999
>>>>>
>>>>> that feels too high, as example we add "List-Unsubscribe" headers in case of ordiany newsletters to support MUA
>>>>> which read that header (for TB a extension exists)
>>>
>>> i have seen that rule hit one customers newsletter generated here, going through a relay responsible for the
>>> sender domain and coming back to the customer itself why it makes it through SA
>>
>> Important question: are those messages being scored as spam?
>>
>> If not, how close to spam (5.0) are they being scored?
>> Are you willing to provide a sample of one such?
>
> in that case not, but depends on some wired HTML, bad chosen
> subject or so hit another rule where 2 points penalty may
> come closer to spam, keep in mind nobody knows how good or
> bad a bayes on the RCPT side maybe trained
>
> what alarmed me was not that our SA in that specific case
> had -1 and not -3 but that a message with a header added
> by good intentions get a high penalty in general
2 points is not a "high penalty". 4+ points is a "high penalty".
Regardless, I've reviewed the recent performance and adjusted the score
limit a bit.
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
Watch... Wallet... Gun... Knee... -- Denny Crane
-----------------------------------------------------------------------
Tomorrow: the 227th anniversary of the signing of the U.S. Constitution
Re: LIST_PARTIAL
Posted by Reindl Harald <h....@thelounge.net>.
Am 16.09.2014 um 16:01 schrieb John Hardin:
> On Tue, 16 Sep 2014, Reindl Harald wrote:
>>> Le 16/09/2014 12:24, Reindl Harald a écrit :
>>>> score LIST_PARTIAL 2.000 1.999 2.000 1.999
>>>>
>>>> that feels too high, as example we add "List-Unsubscribe" headers in case of ordiany newsletters to support MUA
>>>> which read that header (for TB a extension exists)
>>
>> i have seen that rule hit one customers newsletter generated here, going through a relay responsible for the
>> sender domain and coming back to the customer itself why it makes it through SA
>
> Important question: are those messages being scored as spam?
>
> If not, how close to spam (5.0) are they being scored?
> Are you willing to provide a sample of one such?
in that case not, but depends on some wired HTML, bad chosen
subject or so hit another rule where 2 points penalty may
come closer to spam, keep in mind nobody knows how good or
bad a bayes on the RCPT side maybe trained
what alarmed me was not that our SA in that specific case
had -1 and not -3 but that a message with a header added
by good intentions get a high penalty in general
seeing other 100% spam make it through the milter because the
"score RP_MATCHES_RCVD -3.5" (reduced here to -0.5) and so even
outbeating BAYES_99 makes +2 simply too high
Re: LIST_PARTIAL
Posted by John Hardin <jh...@impsec.org>.
On Tue, 16 Sep 2014, Reindl Harald wrote:
>> Le 16/09/2014 12:24, Reindl Harald a écrit :
>>> score LIST_PARTIAL 2.000 1.999 2.000 1.999
>>>
>>> that feels too high, as example we add "List-Unsubscribe" headers in
>>> case of ordiany newsletters to support MUA which read that header (for
>>> TB a extension exists)
>
> i have seen that rule hit one customers newsletter generated here, going
> through a relay responsible for the sender domain and coming back to the
> customer itself why it makes it through SA
Important question: are those messages being scored as spam?
If not, how close to spam (5.0) are they being scored?
Are you willing to provide a sample of one such?
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
Maxim V: Close air support and friendly fire should be easier to
tell apart.
-----------------------------------------------------------------------
Tomorrow: the 227th anniversary of the signing of the U.S. Constitution
Re: LIST_PARTIAL
Posted by Antony Stone <An...@spamassassin.open.source.it>.
On Tuesday 16 September 2014 at 14:07:18 (EU time), John Wilcock wrote:
> Le 16/09/2014 13:29, Reindl Harald a écrit :
> > works, however, the penalty of 2 for 'List-Unsubscribe' without 'List-Id'
> > feels a little bit unfair
>
> What's unfair about being penalised for not being standards-compliant?
>
> RFC2919 states that a mailing list SHOULD add a List-Id header, just as
> RFC2369 states that it SHOULD add a List-Unsubscribe header.
However, in RFC terminology, SHOULD means "ought to, although it's not
compulsory", whereas MUST means "you have to".
Therefore lists which don't add these headers are still RFC compliant.
Antony.
--
Abandon hope, all ye who enter here.
You'll feel much better about things once you do.
Please reply to the list;
please *don't* CC me.
Re: LIST_PARTIAL
Posted by "Kevin A. McGrail" <KM...@PCCC.com>.
On 9/16/2014 8:16 AM, Reindl Harald wrote:
> Am 16.09.2014 um 14:07 schrieb John Wilcock:
>> Le 16/09/2014 13:29, Reindl Harald a écrit :
>>> works, however, the penalty of 2 for 'List-Unsubscribe' without 'List-Id'
>>> feels a little bit unfair
>> What's unfair about being penalised for not being standards-compliant?
>>
>> RFC2919 states that a mailing list SHOULD add a List-Id header,
>> just as RFC2369 states that it SHOULD add a List-Unsubscribe header
> i am talking about a *newsletter* not a classical mailing-list
> and the word SHOULD in a RFC has a different meaning than MUST
> hence a prenalty of 2 is large
>
> the point is that 'List-Unsubscribe' in context of a newsletter
> with modern MUA's is helpful even if it is not a mailing-list
> because it can be displayed in a standard way *additional* to
> the mandatory unscubscribe-link you need to seek in the message
>
> so somebody try to do the enduser something good (as i did) and
> add this header without invent a "list-id" get a *high* penalty
>
> that's how things are if you have both jobs - mailadmin and
> developer of CMS systems with newsletters and try to make both
> jobs as good as possible - you see both sides
>
As a general rule, I only focus efforts on rule rescoring except where
FPs occur or they are likely to occur. This is an edge case at best and
it wasn't causing a FP under SAs default 5.0 scoring.
regards,
KAM
Re: LIST_PARTIAL
Posted by Reindl Harald <h....@thelounge.net>.
Am 16.09.2014 um 14:07 schrieb John Wilcock:
> Le 16/09/2014 13:29, Reindl Harald a écrit :
>> works, however, the penalty of 2 for 'List-Unsubscribe' without 'List-Id'
>> feels a little bit unfair
>
> What's unfair about being penalised for not being standards-compliant?
>
> RFC2919 states that a mailing list SHOULD add a List-Id header,
> just as RFC2369 states that it SHOULD add a List-Unsubscribe header
i am talking about a *newsletter* not a classical mailing-list
and the word SHOULD in a RFC has a different meaning than MUST
hence a prenalty of 2 is large
the point is that 'List-Unsubscribe' in context of a newsletter
with modern MUA's is helpful even if it is not a mailing-list
because it can be displayed in a standard way *additional* to
the mandatory unscubscribe-link you need to seek in the message
so somebody try to do the enduser something good (as i did) and
add this header without invent a "list-id" get a *high* penalty
that's how things are if you have both jobs - mailadmin and
developer of CMS systems with newsletters and try to make both
jobs as good as possible - you see both sides
Re: LIST_PARTIAL
Posted by John Wilcock <jo...@tradoc.fr>.
Le 16/09/2014 13:29, Reindl Harald a écrit :
> works, however, the penalty of 2 for 'List-Unsubscribe' without 'List-Id'
> feels a little bit unfair
What's unfair about being penalised for not being standards-compliant?
RFC2919 states that a mailing list SHOULD add a List-Id header, just as
RFC2369 states that it SHOULD add a List-Unsubscribe header.
--
John
Re: LIST_PARTIAL
Posted by Reindl Harald <h....@thelounge.net>.
Am 16.09.2014 um 12:51 schrieb Reindl Harald:
> Am 16.09.2014 um 12:46 schrieb John Wilcock:
>> I have a low-scored local rule for List-Unsubscribe: without List-Id:, and it does indeed see hits on legit
>> newsletters as well as plenty of not-so-legit ones. But not one of the legit newsletters also hits LIST_PARTIAL.
>
> i have seen that rule hit one customers newsletter generated here, going through
> a relay responsible for the sender domain and coming back to the customer itself
> why it makes it through SA
>
> the header is generated like below by giving the function the unsbcribe URL
> which is anyways in the newsletter itself to support MUA's display it in a
> standard way without seek in the message and so with real good intention
>
> if(!empty($unsubscribe_url))
> {
> $phpmailer->AddCustomHeader('List-Unsubscribe: <' . $unsubscribe_url . '>');
> }
>
> well, i will extend that with
> $phpmailer->AddCustomHeader('List-Id: <' . $customer_domain . '>');
works, however, the penalty of 2 for 'List-Unsubscribe' without 'List-Id'
feels a little bit unfair because without try to support MUA's you won't
become it in that case
BEFORE: -1 BAYES_00,CUST_DNSWL_8,HTML_MESSAGE,LIST_PARTIAL,SPF_PASS
NOW: -3 BAYES_00,CUST_DNSWL_8,HTML_MESSAGE,SPF_PASS
Re: LIST_PARTIAL
Posted by Reindl Harald <h....@thelounge.net>.
Am 16.09.2014 um 12:46 schrieb John Wilcock:
> Le 16/09/2014 12:24, Reindl Harald a écrit :
>> score LIST_PARTIAL 2.000 1.999 2.000 1.999
>>
>> that feels too high, as example we add "List-Unsubscribe"
>> headers in case of ordiany newsletters to support MUA
>> which read that header (for TB a extension exists)
>>
>> IMHO that penalty hits senders which try to be responsible
>
> If the rule were to hit only messages that have List-Unsubscribe: but not List-Id: it would indeed be too high, but
> you'll see that there are various other components to exclude legitimate messages from the the META rule.
>
> I have a low-scored local rule for List-Unsubscribe: without List-Id:, and it does indeed see hits on legit
> newsletters as well as plenty of not-so-legit ones. But not one of the legit newsletters also hits LIST_PARTIAL.
i have seen that rule hit one customers newsletter generated here, going through
a relay responsible for the sender domain and coming back to the customer itself
why it makes it through SA
the header is generated like below by giving the function the unsbcribe URL
which is anyways in the newsletter itself to support MUA's display it in a
standard way without seek in the message and so with real good intention
if(!empty($unsubscribe_url))
{
$phpmailer->AddCustomHeader('List-Unsubscribe: <' . $unsubscribe_url . '>');
}
well, i will extend that with
$phpmailer->AddCustomHeader('List-Id: <' . $customer_domain . '>');
Re: LIST_PARTIAL
Posted by John Wilcock <jo...@tradoc.fr>.
Le 16/09/2014 12:24, Reindl Harald a écrit :
> score LIST_PARTIAL 2.000 1.999 2.000 1.999
>
> that feels too high, as example we add "List-Unsubscribe"
> headers in case of ordiany newsletters to support MUA
> which read that header (for TB a extension exists)
>
> IMHO that penalty hits senders which try to be responsible
If the rule were to hit only messages that have List-Unsubscribe: but
not List-Id: it would indeed be too high, but you'll see that there are
various other components to exclude legitimate messages from the the
META rule.
I have a low-scored local rule for List-Unsubscribe: without List-Id:,
and it does indeed see hits on legit newsletters as well as plenty of
not-so-legit ones. But not one of the legit newsletters also hits
LIST_PARTIAL.
--
John