You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Philip Prindeville <ph...@redfish-solutions.com> on 2015/12/29 21:38:47 UTC
Omitting leading whitespace on headers?
Is there a reason that headers are left with leading spaces?
I’ve noticed that I have to write rules as:
Subject =~ /^ Great [Jj]ob [Oo]pportunity/
because of the leading space… Given the text of RFC-2822:
NO-WS-CTL = %d1-8 / ; US-ASCII control characters
%d11 / ; that do not include the
%d12 / ; carriage return, line feed,
%d14-31 / ; and white space characters
%d127
text = %d1-9 / ; Characters excluding CR and LF
%d11 /
%d12 /
%d14-127 /
obs-text
FWS = ([*WSP CRLF] 1*WSP) / ; Folding white space
obs-FWS
obs-FWS = 1*WSP *(CRLF 1*WSP)
utext = NO-WS-CTL / ; Non white space controls
%d33-126 / ; The rest of US-ASCII
obs-utext
unstructured = *([FWS] utext) [FWS]
subject = "Subject:" unstructured CRLF
Might we consider dropping the first instance of “FWS” preceding the first instance of “utext” in “unstructured”?
-Philip
Re: Omitting leading whitespace on headers?
Posted by Reindl Harald <h....@thelounge.net>.
Am 29.12.2015 um 21:46 schrieb Philip Prindeville:
>
> On Dec 29, 2015, at 1:42 PM, Kevin A. McGrail <KM...@PCCC.com> wrote:
>
>> On 12/29/2015 3:38 PM, Philip Prindeville wrote:
>>> Is there a reason that headers are left with leading spaces?
>>>
>>> I’ve noticed that I have to write rules as:
>>>
>>> Subject =~ /^ Great [Jj]ob [Oo]pportunity/
>>>
>>> because of the leading space…
>> I'm at a complete loss. I add plenty of Subject rules with no leading space. Never seen this issue.
>
>
> I had some rules which weren’t firing so I had to change to /^ .../ or else /^ ?.../ to make them match.
>
> Not sure why.
>
> This is with SA 3.4.1 on Fedora 21
nope, running subject filters from F20 to F23
see example rule from my first answer
what says "spamassassin --lint"?
Re: Omitting leading whitespace on headers?
Posted by "Kevin A. McGrail" <KM...@PCCC.com>.
On 12/29/2015 5:16 PM, Philip Prindeville wrote:
>
> On Dec 29, 2015, at 3:15 PM, Kevin A. McGrail <KMcGrail@PCCC.com
> <ma...@PCCC.com>> wrote:
>
>> On 12/29/2015 5:12 PM, Philip Prindeville wrote:
>>> I did recall that I used the patch here:
>>>
>>> https://bz.apache.org/SpamAssassin/show_bug.cgi?id=6360#c4
>>>
>>> to be able to debug my rules, using a rule that would match any non-empty subject: value to dump out what it was (the “====> got hit: “…”” line), and it was always showing a leading space…
>>>
>>> -Philip
>>>
>>
>> Thank goodness. You had me worried we have something foundational
>> processing issue!
>>
>> Regards,
>> KAM
>>
>
> No, I eventually added “^ ?” to all of my Subject rules… but I’m
> thinking I shouldn’t have had to.
>
You definitely shouldn't have to. You should try a clean install if you
can.
Regards,
KAM
Re: Omitting leading whitespace on headers?
Posted by Philip Prindeville <ph...@redfish-solutions.com>.
On Dec 29, 2015, at 3:15 PM, Kevin A. McGrail <KM...@PCCC.com> wrote:
> On 12/29/2015 5:12 PM, Philip Prindeville wrote:
>> I did recall that I used the patch here:
>>
>> https://bz.apache.org/SpamAssassin/show_bug.cgi?id=6360#c4
>>
>> to be able to debug my rules, using a rule that would match any non-empty subject: value to dump out what it was (the “====> got hit: “…”” line), and it was always showing a leading space…
>>
>> -Philip
>>
>
> Thank goodness. You had me worried we have something foundational processing issue!
>
> Regards,
> KAM
>
No, I eventually added “^ ?” to all of my Subject rules… but I’m thinking I shouldn’t have had to.
Re: Omitting leading whitespace on headers?
Posted by "Kevin A. McGrail" <KM...@PCCC.com>.
On 12/29/2015 5:12 PM, Philip Prindeville wrote:
> I did recall that I used the patch here:
>
> https://bz.apache.org/SpamAssassin/show_bug.cgi?id=6360#c4
>
> to be able to debug my rules, using a rule that would match any non-empty subject: value to dump out what it was (the “====> got hit: “…”” line), and it was always showing a leading space…
>
> -Philip
>
Thank goodness. You had me worried we have something foundational
processing issue!
Regards,
KAM
--
*Kevin A. McGrail*
CEO
Peregrine Computer Consultants Corporation
3927 Old Lee Highway, Suite 102-C
Fairfax, VA 22030-2422
http://www.pccc.com/
703-359-9700 x50 / 800-823-8402 (Toll-Free)
703-798-0171 (wireless)
KMcGrail@PCCC.com <ma...@pccc.com>
Re: Omitting leading whitespace on headers?
Posted by Philip Prindeville <ph...@redfish-solutions.com>.
On Dec 29, 2015, at 2:39 PM, Kevin A. McGrail <KM...@PCCC.com> wrote:
> On 12/29/2015 4:29 PM, Philip Prindeville wrote:
>> On Dec 29, 2015, at 2:14 PM, Kevin A. McGrail <KM...@PCCC.com> wrote:
>>
>>> On 12/29/2015 3:46 PM, Philip Prindeville wrote:
>>>> On Dec 29, 2015, at 1:42 PM, Kevin A. McGrail <KM...@PCCC.com> wrote:
>>>>
>>>>> On 12/29/2015 3:38 PM, Philip Prindeville wrote:
>>>>>> Is there a reason that headers are left with leading spaces?
>>>>>>
>>>>>> I’ve noticed that I have to write rules as:
>>>>>>
>>>>>> Subject =~ /^ Great [Jj]ob [Oo]pportunity/
>>>>>>
>>>>>> because of the leading space…
>>>>> I'm at a complete loss. I add plenty of Subject rules with no leading space. Never seen this issue.
>>>> I had some rules which weren’t firing so I had to change to /^ .../ or else /^ ?.../ to make them match.
>>>>
>>>> Not sure why.
>>>>
>>>> This is with SA 3.4.1 on Fedora 21.
>>>>
>>>> -Philip
>>> What's the original Subject header look like from the original mail?
>>>
>>> Regards,
>>> KAM
>>
>> This was a while ago. I’d have to go back and look. Maybe this one?
>>
>> Subject: [IDN][#2056301] CareerBuilder: Open position for you
> OK, I was thinking perhaps an alternate charset or something but never run into this issue.
>
> If you are anchoring your Subject searches, allowing for whitespace, etc. is a decent idea though from Reindl.
>
> regards,
> KAM
I did recall that I used the patch here:
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=6360#c4
to be able to debug my rules, using a rule that would match any non-empty subject: value to dump out what it was (the “====> got hit: “…”” line), and it was always showing a leading space…
-Philip
Re: Omitting leading whitespace on headers?
Posted by "Kevin A. McGrail" <KM...@PCCC.com>.
On 12/29/2015 4:29 PM, Philip Prindeville wrote:
> On Dec 29, 2015, at 2:14 PM, Kevin A. McGrail <KM...@PCCC.com> wrote:
>
>> On 12/29/2015 3:46 PM, Philip Prindeville wrote:
>>> On Dec 29, 2015, at 1:42 PM, Kevin A. McGrail <KM...@PCCC.com> wrote:
>>>
>>>> On 12/29/2015 3:38 PM, Philip Prindeville wrote:
>>>>> Is there a reason that headers are left with leading spaces?
>>>>>
>>>>> I’ve noticed that I have to write rules as:
>>>>>
>>>>> Subject =~ /^ Great [Jj]ob [Oo]pportunity/
>>>>>
>>>>> because of the leading space…
>>>> I'm at a complete loss. I add plenty of Subject rules with no leading space. Never seen this issue.
>>> I had some rules which weren’t firing so I had to change to /^ .../ or else /^ ?.../ to make them match.
>>>
>>> Not sure why.
>>>
>>> This is with SA 3.4.1 on Fedora 21.
>>>
>>> -Philip
>> What's the original Subject header look like from the original mail?
>>
>> Regards,
>> KAM
>
> This was a while ago. I’d have to go back and look. Maybe this one?
>
> Subject: [IDN][#2056301] CareerBuilder: Open position for you
OK, I was thinking perhaps an alternate charset or something but never
run into this issue.
If you are anchoring your Subject searches, allowing for whitespace,
etc. is a decent idea though from Reindl.
regards,
KAM
Re: Omitting leading whitespace on headers?
Posted by Philip Prindeville <ph...@redfish-solutions.com>.
On Dec 29, 2015, at 2:14 PM, Kevin A. McGrail <KM...@PCCC.com> wrote:
> On 12/29/2015 3:46 PM, Philip Prindeville wrote:
>> On Dec 29, 2015, at 1:42 PM, Kevin A. McGrail <KM...@PCCC.com> wrote:
>>
>>> On 12/29/2015 3:38 PM, Philip Prindeville wrote:
>>>> Is there a reason that headers are left with leading spaces?
>>>>
>>>> I’ve noticed that I have to write rules as:
>>>>
>>>> Subject =~ /^ Great [Jj]ob [Oo]pportunity/
>>>>
>>>> because of the leading space…
>>> I'm at a complete loss. I add plenty of Subject rules with no leading space. Never seen this issue.
>>
>> I had some rules which weren’t firing so I had to change to /^ .../ or else /^ ?.../ to make them match.
>>
>> Not sure why.
>>
>> This is with SA 3.4.1 on Fedora 21.
>>
>> -Philip
> What's the original Subject header look like from the original mail?
>
> Regards,
> KAM
This was a while ago. I’d have to go back and look. Maybe this one?
Subject: [IDN][#2056301] CareerBuilder: Open position for you
Re: Omitting leading whitespace on headers?
Posted by Reindl Harald <h....@thelounge.net>.
Am 29.12.2015 um 22:14 schrieb Kevin A. McGrail:
> On 12/29/2015 3:46 PM, Philip Prindeville wrote:
>> On Dec 29, 2015, at 1:42 PM, Kevin A. McGrail <KM...@PCCC.com> wrote:
>>
>>> On 12/29/2015 3:38 PM, Philip Prindeville wrote:
>>>> Is there a reason that headers are left with leading spaces?
>>>>
>>>> I’ve noticed that I have to write rules as:
>>>>
>>>> Subject =~ /^ Great [Jj]ob [Oo]pportunity/
>>>>
>>>> because of the leading space…
>>> I'm at a complete loss. I add plenty of Subject rules with no
>>> leading space. Never seen this issue.
>>
>> I had some rules which weren’t firing so I had to change to /^ .../ or
>> else /^ ?.../ to make them match.
>>
>> Not sure why.
>>
>> This is with SA 3.4.1 on Fedora 21.
>>
> What's the original Subject header look like from the original mail?
with "normalize_charset 1" and replace whitespaces in rules with "\s+"
one can't trick SA with leading or multiple whitespaces in the junkmail
frankly with body rules i have seen even cases where a line was split in
two html cells and the SA rule hits because the junk is normalized
before proceed it against rules
Re: Omitting leading whitespace on headers?
Posted by "Kevin A. McGrail" <KM...@PCCC.com>.
On 12/29/2015 3:46 PM, Philip Prindeville wrote:
> On Dec 29, 2015, at 1:42 PM, Kevin A. McGrail <KM...@PCCC.com> wrote:
>
>> On 12/29/2015 3:38 PM, Philip Prindeville wrote:
>>> Is there a reason that headers are left with leading spaces?
>>>
>>> I’ve noticed that I have to write rules as:
>>>
>>> Subject =~ /^ Great [Jj]ob [Oo]pportunity/
>>>
>>> because of the leading space…
>> I'm at a complete loss. I add plenty of Subject rules with no leading space. Never seen this issue.
>
> I had some rules which weren’t firing so I had to change to /^ .../ or else /^ ?.../ to make them match.
>
> Not sure why.
>
> This is with SA 3.4.1 on Fedora 21.
>
> -Philip
What's the original Subject header look like from the original mail?
Regards,
KAM
Re: Omitting leading whitespace on headers?
Posted by Philip Prindeville <ph...@redfish-solutions.com>.
On Dec 29, 2015, at 1:42 PM, Kevin A. McGrail <KM...@PCCC.com> wrote:
> On 12/29/2015 3:38 PM, Philip Prindeville wrote:
>> Is there a reason that headers are left with leading spaces?
>>
>> I’ve noticed that I have to write rules as:
>>
>> Subject =~ /^ Great [Jj]ob [Oo]pportunity/
>>
>> because of the leading space…
> I'm at a complete loss. I add plenty of Subject rules with no leading space. Never seen this issue.
I had some rules which weren’t firing so I had to change to /^ .../ or else /^ ?.../ to make them match.
Not sure why.
This is with SA 3.4.1 on Fedora 21.
-Philip
Re: Omitting leading whitespace on headers?
Posted by "Kevin A. McGrail" <KM...@PCCC.com>.
On 12/29/2015 3:38 PM, Philip Prindeville wrote:
> Is there a reason that headers are left with leading spaces?
>
> I’ve noticed that I have to write rules as:
>
> Subject =~ /^ Great [Jj]ob [Oo]pportunity/
>
> because of the leading space…
I'm at a complete loss. I add plenty of Subject rules with no leading
space. Never seen this issue.
Re: Omitting leading whitespace on headers?
Posted by Reindl Harald <h....@thelounge.net>.
Am 29.12.2015 um 21:38 schrieb Philip Prindeville:
> Is there a reason that headers are left with leading spaces?
>
> I’ve noticed that I have to write rules as:
>
> Subject =~ /^ Great [Jj]ob [Oo]pportunity/
>
> because of the leading space… Given the text of RFC-2822
no, we have a ton of subject and body rules and the work fine without a
leading whitespace
maybe "normalize_charset 1" makes the difference but we had them AFAIR
also before enable that option
# Subject Begins High
header __CUST_SUBJ_9 Subject =~
/^(apply\s+for\s+urgent\s+loan|auslieferungsankündigung\s+betrefend\s+ihre\s+sendung|deutsche\s+bank\:\s+sicherheitssperre\s+ihres\s+kontos|festnetz\-rechnung|have\s+you\s+won|ihr\s+konto\s+wurde\s+begrenzt|ihre\s+festnetz\-rechnung|ihre\s+mobilfunk\-rechnung|investment\s+opportunit|investment\s+quest|kredit|loan|mobilfunk\-rechnung|neue\s+festnetz\-rechnung|neue\s+mobilfunk\-rechnung|proposal|sie\s+haben\s+gewonnen|verify\s+your\s+account|you\s+won|your\s+paypal\s+account\s+has\s+been\s+limited|\[paypal\]\s+check\s+the\s+account\s+paypal).*/i
meta CUST_SUBJ_9 (__HAS_SUBJECT && __CUST_SUBJ_9)
score CUST_SUBJ_9 3.5
describe CUST_SUBJ_9 Begins High
[roo[root@mail-gw:~]$ cat maillog | grep CUST_SUBJ_9 | wc -l
16t@mail-gw:~]$ cat maillog | grep CUST_SUBJ_9 | wc -l
16