You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spamassassin.apache.org by bu...@spamassassin.apache.org on 2021/05/20 11:10:34 UTC
[Bug 7910] New: Incorrect Thread-index in zimbra formed messages
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7910
Bug ID: 7910
Summary: Incorrect Thread-index in zimbra formed messages
Product: Spamassassin
Version: 3.4.6
Hardware: PC
OS: Windows NT
Status: NEW
Severity: normal
Priority: P2
Component: Rules
Assignee: dev@spamassassin.apache.org
Reporter: informatique@qc.bzh
Target Milestone: Undefined
Hello,
Our sended mails trigger the PDS_BAD_THREAD_QP_64 rule.
It seems that the header Thread-Index is malformed.
As far I know, the rule is correct for exchange servers sending messages. But
it seems that zimbra is using the header in an other way.
The current definition in 72_active.cf is :
header __THREAD_INDEX_GOOD Thread-Index =~
m,^A[a-z0-9][A-Za-z0-9+/]{27}(?:[A-Za-z0-9+/]{20})?(?:[AQgw]==|[A-Za-z0-9+/]{7}|[A-Za-z0-9+/]{13}[AEIMQUYcgkosw048]=)$,
Here is an example header our server is generating :
Thread-Index: TBTkM9019y9w+zdGmptIaSuZKsAA7g==
I can find such "malformed" headers in official zimbra documentation, so I
presume it is not due to a bad configuration.
Thank you for checking.
--
You are receiving this mail because:
You are the assignee for the bug.
Re: [Bug 7910] Incorrect Thread-index in zimbra formed messages
Posted by Jared Hall <ja...@jaredsec.com>.
On 1/18/2023 10:02 AM, bugzilla-daemon@spamassassin.apache.org wrote:
> https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7910
>
> Bill Cole<bi...@apache.org> changed:
>
> What |Removed |Added
> ----------------------------------------------------------------------------
> Status|NEW |RESOLVED
> Resolution|--- |INVALID
> CC| |billcole@apache.org
>
> --- Comment #2 from Bill Cole<bi...@apache.org> ---
> PDS_BAD_THREAD_QP_64 is not in the current SpamAssassin ruleset.
>
> mail-tester.com is not part of the ASF SpamAssassin project and they are known
> to be using an obsolete version with rules and scores that do not match the
> current SA default ruleset.
>
> Take any problems they claim to exist in your mail to them. We cannot answer
> for their failure to provide accurate results.
>
1) Correct. From the Zimbra Config Guide:
1160 zimbraMailThreadingAlgorithm enum 8.0.0 The algorithm
to use when aggregating new messages into conversations. Possible values
are: - "none": no conversation threading is performed. - "subject": the
message will be threaded based solely on its normalized subject. -
"strict": only the threading message headers (References, In-Reply-To,
Message-ID, and Resent-Message-ID) are used to correlate messages. No
checking of normalized subjects is performed. - "references": the same
logic as "strict" with the constraints slightly altered so that the
non-standard Thread-Index header is considered when threading messages
and that a reply message lacking References and In-Reply-To headers will
fall back to using subject-based threading. - "subjrefs": the same logic
as "references" with the further caveat that changes in the normalized
subject will break a thread in two.
I believe the main RFC that deals with Message Threading correlates to
Zimbra's "strict" mode, particularly the In-Reply-To and References
headers. Where Zimbra overstepped is that while there is no RFC
describing Thread-Index, there IS a standard. Microsoft first started
using this back in 2000, and documented it within MSDN in 2003. Zimbra
simply chose to do their own thing (whatever that may be).
2) Don't want to be a party-pooper, but the veracity of the underlying
regex (__THREAD_INDEX_GOOD) is suspect. I found several hundred
Thread-Index headers and found many that fail with standard, non-spam,
Outlook messages. Here are a couple (Outlook 2007 and Outlook 2013):
X-Mailer: Microsoft Outlook 14.0
Thread-Index: AQLYgMQp8BCX9nSr6CIOlZIDeiK7/KkngMmA
X-Mailer: Microsoft Outlook 16.0
Thread-Index: AQH+vjdk9tD1ot0pJqzGHeFq00ZfdgFRP4zA
Notice the UPPERCASE second character. I've also found a lot of hosted
Exchange and Exchange Online headers that don't match this regex. Those
are much longer strings.
3) You *might* want to use something like this for __THREAD_INDEX_GOOD:
header __THREAD_INDEX_GOOD Thread-Index =~ m,^(A[A-z0-9][A-z0-9+\/]{27}(?:[A-z0-9+\/]{20})?(?:[AQgw]==|[A-z0-9+\/]{7}|[A-z0-9+\/]{13}[AEIMQUYcgkosw048]=)|[A-z0-9\/+]{30}==)$,
This addresses the Outlook headers as indicated above. It also provides
support for Zimbra headers (last [A-z0-9\/+]{30}== grouping).
4) While it is good that PDS_BAD_THREAD_QP_64 is dead, if
__THREAD_INDEX_GOOD is loose, then so are its dependencies:
__ADVANCE_FEE_5_NEW, __ADVANCE_FEE_4_NEW, __ADVANCE_FEE_3_NEW,
__ADVANCE_FEE_2_NEW, TO_NO_BRKTS_PCNT, TO_NO_BRKTS_MSFT, TO_IN_SUBJ,
TO_EQ_FM_DIRECT_MX, MONEY_FORM_SHORT, HAS_X_OUTGOING_SPAM_STAT (as
observed in SA v3.4.6).
Or....kill off __THREAD_INDEX_GOOD entirely. It has to be hard to
maintain header support like that.
$0.02,
-- Jared Hall
[Bug 7910] Incorrect Thread-index in zimbra formed messages
Posted by bu...@spamassassin.apache.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7910
Bill Cole <bi...@apache.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |RESOLVED
Resolution|--- |INVALID
CC| |billcole@apache.org
--- Comment #2 from Bill Cole <bi...@apache.org> ---
PDS_BAD_THREAD_QP_64 is not in the current SpamAssassin ruleset.
mail-tester.com is not part of the ASF SpamAssassin project and they are known
to be using an obsolete version with rules and scores that do not match the
current SA default ruleset.
Take any problems they claim to exist in your mail to them. We cannot answer
for their failure to provide accurate results.
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 7910] Incorrect Thread-index in zimbra formed messages
Posted by bu...@spamassassin.apache.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7910
Benny Pedersen <me...@junc.eu> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |me@junc.eu
--- Comment #3 from Benny Pedersen <me...@junc.eu> ---
remember to get support one need to use sa-update, no support if this is not
runned in cron installation of zimbra precompiled problems
same goes for mail-tester
--
You are receiving this mail because:
You are the assignee for the bug.
Re: [Bug 7910] Incorrect Thread-index in zimbra formed messages
Posted by Michael Peddemors <mi...@linuxmagic.com>.
It's also a pain when Zimbra injects the SA headers ABOVE the trace
headers, when it should be added to the bottom of the headers.
On 2023-01-18 01:24, bugzilla-daemon@spamassassin.apache.org wrote:
> https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7910
>
> SysNet Team <sy...@quantificare.com> changed:
>
> What |Removed |Added
> ----------------------------------------------------------------------------
> CC| |sysnet@quantificare.com
>
> --- Comment #1 from SysNet Team <sy...@quantificare.com> ---
> Hello,
>
> We have the similar issue with our Zimbra mail server. We already contacted the
> support and they said to us that the issue was related here. No issue on Zimbra
> side.
>
> Our sent mails still trigger the PDS_BAD_THREAD_QP_64 rule. The header is still
> malformed. We tested our mails with the tool called mail-tester :
> https://www.mail-tester.com/
>
> All of us mails trigger this rule and we lost rate because of it. As the bug is
> open since almost 2 years now, do you have some updates / advice for us ?
>
> Thank you.
>
--
"Catch the Magic of Linux..."
------------------------------------------------------------------------
Michael Peddemors, President/CEO LinuxMagic Inc.
Visit us at http://www.linuxmagic.com @linuxmagic
A Wizard IT Company - For More Info http://www.wizard.ca
"LinuxMagic" a Registered TradeMark of Wizard Tower TechnoServices Ltd.
------------------------------------------------------------------------
604-682-0300 Beautiful British Columbia, Canada
This email and any electronic data contained are confidential and intended
solely for the use of the individual or entity to which they are addressed.
Please note that any views or opinions presented in this email are solely
those of the author and are not intended to represent those of the company.
[Bug 7910] Incorrect Thread-index in zimbra formed messages
Posted by bu...@spamassassin.apache.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7910
SysNet Team <sy...@quantificare.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |sysnet@quantificare.com
--- Comment #1 from SysNet Team <sy...@quantificare.com> ---
Hello,
We have the similar issue with our Zimbra mail server. We already contacted the
support and they said to us that the issue was related here. No issue on Zimbra
side.
Our sent mails still trigger the PDS_BAD_THREAD_QP_64 rule. The header is still
malformed. We tested our mails with the tool called mail-tester :
https://www.mail-tester.com/
All of us mails trigger this rule and we lost rate because of it. As the bug is
open since almost 2 years now, do you have some updates / advice for us ?
Thank you.
--
You are receiving this mail because:
You are the assignee for the bug.