You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Eddy Beliveau <ed...@hec.ca> on 2009/01/08 22:05:42 UTC
Malformed UTF-8 character with SA 3.2.5
Hi!
I sent this email to the amavisd-new group but didn't received any replies
I give it a spin on this group
Maybe someone can help
Thanks in advance
Eddy
----------------- original email
Hi!
We are using Postfix 2.5.5 on our RHEL AS release 4 (Nahant Update 6)
academic server.
amavisd-new-2.6.2 (20081215), Unicode aware, LANG="fr"
with spamassassin 3.2.5
Our log file contains many:
amavis[19738]: (19738-05) _WARN: Malformed UTF-8 character (unexpected
continuation byte 0x8e, with no preceding start byte) in pattern match
(m//) at
/var/lib/spamassassin/3.002005/70_sare_specific_cf_sare_sa-update_dostech_net/200605280300.cf,
rule SARE_SPEC_REPL_OBFU2, line 1, <GEN16> line 3620.
That rule is version 01.03.13
Can someone help ?
Many thanks
Eddy
--
Eddy Beliveau
HEC Montreal
Montreal (Quebec)
Canada
Re: Malformed UTF-8 character with SA 3.2.5
Posted by Eddy Beliveau <ed...@hec.ca>.
Hi! Mark,
Many thanks for your reply.
I also have similar WARN error
Jan 9 09:20:02 smtpext2 amavis[20636]: (20636-16) _WARN: Malformed
UTF-8 character (unexpected continuation byte 0xba, with no preceding
start byte) in pattern match (m//) at
/usr/lib/perl5/site_perl/5.8.5/Mail/SpamAssassin/Plugin/BodyEval.pm line
243, <GEN16> line 10082.
As suggested, I created my first spamassassin bug report (6042) and
include a sample
Thanks,
Eddy
-------- Message original --------
Sujet : Re: Malformed UTF-8 character with SA 3.2.5
De : Mark Martinec <Ma...@ijs.si>
Pour : users@spamassassin.apache.org
Date : 2009-01-08 18:55
> Eddy,
>
>> I sent this email to the amavisd-new group but didn't received any replies
>> I give it a spin on this group
>> Maybe someone can help
>
> Yes, this is probably a more suitable place for this question.
>
>> We are using Postfix 2.5.5 on our RHEL AS release 4 (Nahant Update 6)
>> academic server.
>>
>> amavisd-new-2.6.2 (20081215), Unicode aware, LANG="fr"
>> with spamassassin 3.2.5
>>
>> Our log file contains many:
>> amavis[19738]: (19738-05) _WARN: Malformed UTF-8 character (unexpected
>> continuation byte 0x8e, with no preceding start byte) in pattern match
>> (m//) at
>> /var/lib/spamassassin/3.002005/70_sare_specific_cf_sare_sa-update_dostech_n
>> et/200605280300.cf, rule SARE_SPEC_REPL_OBFU2, line 1, <GEN16> line 3620.
>>
>> That rule is version 01.03.13
>> Can someone help ?
>
> I searched our logs for something similar and came up with a possibly related
> case, but in a different code section. Here is mine (using SA 3.3):
>
> rules: failed to run TVD_STOCK1 test, skipping:
> (Malformed UTF-8 character (fatal)
> at /usr/local/lib/perl5/site_perl/5.10.0/Mail/SpamAssassin/Plugin/BodyEval.pm
> line 250, <GEN11> line 499.
>
> This one is within sub _check_stock_info, evaluating the regexp:
> $rnd_chunk =~ /^\s*([^:\s][^:\n]{2,29})\s*:\s*\S/mg
> on a perfectly valid UTF-8 string. It turns out it is a bug in
> perl5.8.8, 5.8.9 and in 5.10.0 - the bug goes away if the string
> is not tainted.
>
> I filed a Perl bug report yesterday:
> perlbug: [perl #62048]
> Unwarranted "Malformed UTF-8 character" on tainted variable
>
> I don't know if your case of "Malformed UTF-8 character" is due
> to the same bug, or is it a SpamAssassin problem.
>
> Can you make one of your mail samples available for examination?
> With any luck the problem is reproducible, making it much easier
> to resolve or provide a workaround. It's probably best to open
> a SpamAssassin bug case and attach a sample.
>
> Mark
>
--
Eddy Beliveau
HEC Montreal
Montreal (Quebec)
Canada
Re: Malformed UTF-8 character with SA 3.2.5
Posted by Sergey Kovalev <sp...@kovalev.com.ru>.
Mark Martinec wrote:
>
>> Our log file contains many:
>> amavis[19738]: (19738-05) _WARN: Malformed UTF-8 character (unexpected
>> continuation byte 0x8e, with no preceding start byte) in pattern match
>> (m//) at
>> /var/lib/spamassassin/3.002005/70_sare_specific_cf_sare_sa-update_dostech_n
>> et/200605280300.cf, rule SARE_SPEC_REPL_OBFU2, line 1, <GEN16> line 3620.
>
> I searched our logs for something similar and came up with a possibly related
> case, but in a different code section. Here is mine (using SA 3.3):
>
> rules: failed to run TVD_STOCK1 test, skipping:
> (Malformed UTF-8 character (fatal)
> at /usr/local/lib/perl5/site_perl/5.10.0/Mail/SpamAssassin/Plugin/BodyEval.pm
> line 250, <GEN11> line 499.
>
> This one is within sub _check_stock_info, evaluating the regexp:
> $rnd_chunk =~ /^\s*([^:\s][^:\n]{2,29})\s*:\s*\S/mg
> on a perfectly valid UTF-8 string. It turns out it is a bug in
> perl5.8.8, 5.8.9 and in 5.10.0 - the bug goes away if the string
> is not tainted.
I'm not quite familiar with utf-8 handling in perl, but it seems to me
that there are some different flavours of UTF :)
I'm trying to score cyrillic pr0n messages. And it is not simple.
In order SA to start hitting my regexp rules I've modified Check.pm with
`use utf8;' string and added normalize_charset 1 to local.cf.
Now it detects bad words but spamd generates warnings at startup/reload
Jan 9 01:54:40 mx spamd[19814]: Malformed UTF-8 character (unexpected
non-continuation byte 0xe8, immediately after start byte 0xe9) in eval
"string" at
/var/db/spamassassin/3.002005/updates_spamassassin_org/20_advance_fee.cf,
rule __FRAUD_GAN, line 1.
Jan 9 01:54:40 mx spamd[19814]: Malformed UTF-8 character (unexpected
non-continuation byte 0x5d, immediately after start byte 0xe8) in eval
"string" at
/var/db/spamassassin/3.002005/updates_spamassassin_org/20_advance_fee.cf,
rule __FRAUD_GAN, line 1.
BTW, I don't need 20_advance_fee.cf but want to use
updates.spamassassin.org channel. Is it possible to somehow ignore some
*.cf?
Re: Malformed UTF-8 character with SA 3.2.5
Posted by Mark Martinec <Ma...@ijs.si>.
Eddy,
> I sent this email to the amavisd-new group but didn't received any replies
> I give it a spin on this group
> Maybe someone can help
Yes, this is probably a more suitable place for this question.
> We are using Postfix 2.5.5 on our RHEL AS release 4 (Nahant Update 6)
> academic server.
>
> amavisd-new-2.6.2 (20081215), Unicode aware, LANG="fr"
> with spamassassin 3.2.5
>
> Our log file contains many:
> amavis[19738]: (19738-05) _WARN: Malformed UTF-8 character (unexpected
> continuation byte 0x8e, with no preceding start byte) in pattern match
> (m//) at
> /var/lib/spamassassin/3.002005/70_sare_specific_cf_sare_sa-update_dostech_n
>et/200605280300.cf, rule SARE_SPEC_REPL_OBFU2, line 1, <GEN16> line 3620.
>
> That rule is version 01.03.13
> Can someone help ?
I searched our logs for something similar and came up with a possibly related
case, but in a different code section. Here is mine (using SA 3.3):
rules: failed to run TVD_STOCK1 test, skipping:
(Malformed UTF-8 character (fatal)
at /usr/local/lib/perl5/site_perl/5.10.0/Mail/SpamAssassin/Plugin/BodyEval.pm
line 250, <GEN11> line 499.
This one is within sub _check_stock_info, evaluating the regexp:
$rnd_chunk =~ /^\s*([^:\s][^:\n]{2,29})\s*:\s*\S/mg
on a perfectly valid UTF-8 string. It turns out it is a bug in
perl5.8.8, 5.8.9 and in 5.10.0 - the bug goes away if the string
is not tainted.
I filed a Perl bug report yesterday:
perlbug: [perl #62048]
Unwarranted "Malformed UTF-8 character" on tainted variable
I don't know if your case of "Malformed UTF-8 character" is due
to the same bug, or is it a SpamAssassin problem.
Can you make one of your mail samples available for examination?
With any luck the problem is reproducible, making it much easier
to resolve or provide a workaround. It's probably best to open
a SpamAssassin bug case and attach a sample.
Mark