spamassassin --remove-markup and dos newlines

[Ryan Thompson <sp...@sasknow.com>]

Hi all,

In the process of deploying a new server, I just tracked down an
interesting problem with 2.63 (which was about half a dozen problems
deep in our "call stack" of issues to iron out :-)

I noticed that spamassassin -d (and probably sa-learn & co.) isn't able
to fully remove markup from messages stored with dos newlines. A prime
example of this is the cyrus store directory message format, and
possibly other maildir configurations, too. SpamAssassin would correctly
remove the X-Spam-* headers, but would not replace the body report,
which, I'm assuming, is due to apparently illegible MIME headers, when
UNIX newlines are assumed.

I worked around this issue by replacing the DOS newlines before feeding
to spamassassin -d, and the report (and headers) were removed
successfully, as expected.

I guess it's debatable whether this is a bug in SA or not, which is why
I'm posting here for discussion. Would it be generally advantageous for
SpamAssassin to cope with either newline format?

Thanks,
- Ryan

-- 
  Ryan Thompson <ry...@sasknow.com>

  SaskNow Technologies - http://www.sasknow.com
  901-1st Avenue North - Saskatoon, SK - S7K 1Y4

        Tel: 306-664-3600   Fax: 306-244-7037   Saskatoon
  Toll-Free: 877-727-5669     (877-SASKNOW)     North America

Re: spamassassin --remove-markup and dos newlines

[Ryan Thompson <sp...@sasknow.com>]

Loren Wilton wrote to spamassassin-users@incubator.apache.org:

> SA seems to deal well with DOS newlines in most other cases, and there
> is a native-Windows compilable version of SA.
>
> From that, I'd say that SA really should handle dos newlines, if not
> perfectly, at least acceptably well, in this case too.

That makes sense to me, too. If nothing else, this was a nasty side-
effect to track down, especially considering the dos newlines were
generated by a popular UNIX MDA (cyrus-imapd). :-)

I've also confirmed that the same symptom exists with

# spamassassin -V
SpamAssassin version 3.0.0-pre1
  running on Perl version 5.8.2

.. as well as 2.63

- Ryan

-- 
  Ryan Thompson <ry...@sasknow.com>

  SaskNow Technologies - http://www.sasknow.com
  901-1st Avenue North - Saskatoon, SK - S7K 1Y4

        Tel: 306-664-3600   Fax: 306-244-7037   Saskatoon
  Toll-Free: 877-727-5669     (877-SASKNOW)     North America

Re: spamassassin --remove-markup and dos newlines

[Loren Wilton <lw...@earthlink.net>]

SA seems to deal well with DOS newlines in most other cases, and there is a
native-Windows compilable version of SA.

>From that, I'd say that SA really should handle dos newlines, if not
perfectly, at least acceptably well, in this case too.

        Loren

> I noticed that spamassassin -d (and probably sa-learn & co.) isn't able
> to fully remove markup from messages stored with dos newlines. A prime
> example of this is the cyrus store directory message format, and
> possibly other maildir configurations, too. SpamAssassin would correctly
> remove the X-Spam-* headers, but would not replace the body report,
> which, I'm assuming, is due to apparently illegible MIME headers, when
> UNIX newlines are assumed.