You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Clayton Cottingham <cl...@ardishealth.com> on 2008/10/21 23:16:32 UTC
koi8-r ok_locales and ok_languages
hi all
trying to figure a way to filter out, or better yet score charset koi8-r
down some
ok_locales and ok_languages do not work as expected
inactive_languages is a no go as well
any help appreciated
thanks
Re: koi8-r ok_locales and ok_languages
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Tue, 2008-10-21 at 14:16 -0700, Clayton Cottingham wrote:
> hi all
>
> trying to figure a way to filter out, or better yet score charset koi8-r
> down some
ok_locales en # all Western char sets in general
See section Language Options in the docs.
http://spamassassin.apache.org/full/3.2.x/doc/Mail_SpamAssassin_Conf.html
> ok_locales and ok_languages do not work as expected
> inactive_languages is a no go as well
>
> any help appreciated
If you do NOT get any hits on CHARSET_FARAWAY, CHARSET_FARAWAY_HEADER
and MIME_CHARSET_FARAWAY, please provide more details.
What and how did you try exactly that does not work? How do you tell it
does not work? What do you expect, what does not work? Where did you
specify ok_locales, and did you edit the correct file?
guenther
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: koi8-r ok_locales and ok_languages
Posted by Ned Slider <ne...@unixmail.co.uk>.
Clayton Cottingham wrote:
> hi all
>
> trying to figure a way to filter out, or better yet score charset koi8-r
> down some
>
> ok_locales and ok_languages do not work as expected
> inactive_languages is a no go as well
>
> any help appreciated
>
> thanks
>
Here's my custom rule for detecting undesirable charsets in the Subject
field:
header LOCAL_CHARSET_SUBJECT Subject:raw =~
/\=\?(koi8-r|windows-1251|iso-2022-jp|gb2312)\?/i
score LOCAL_CHARSET_SUBJECT 3
describe LOCAL_CHARSET_SUBJECT Contains charsets we don't accept
It hits on about 14% spam for me at present. Adapt to suit the languages
you want to score against. I also wrote a similar rule for rawbody but
it hits on far less.
Re: koi8-r ok_locales and ok_languages
Posted by Clayton Cottingham <cl...@ardishealth.com>.
i linted ever time and never got any errors, with or without the TextCat
plugin
thanks for the tips on the inactive languages
ill post the emails asap
Karsten � wrote:
>> # Set ok_locals to just en
>> ok_locales en
>>
>
> Please note that the comment above is wrong. It actually means
> # all Western char sets in general
>
>
>> # Set ok_languages to just english and french
>> ok_languages en fr
>>
>> # Set inactive_languages all defaults plus russian and korean
>> inactive_languages bs cy eo et eu fy ga gd is ko la lt lv rm ru sa sco sl yi
>>
>
> By that, you disabled Russian and Korean detection, err, guessing. ;)
> The TextCat plugin thus will never detect any mail written in either
> Russian or Korean -- and subsequently will not be able to tell it is an
> UNWANTED_LANGUAGE_BODY, because it didn't guess any language.
>
> In other words: TextCat will not identify text to be written in the
> languages specified by inactive_languages, because you told it to. If it
> can't guess a language with sufficient confidence, it can't be sure it
> is outside the list of ok_languages either.
>
> See the previously referenced documentation for the TextCat plugin.
>
>
>
>> all three were run seperately and together with no difference, aka spam
>> still got through
>>
>> spamassassin was restarted /etc/init.d/spamassassin restart
>>
>
> With "ok_locales en", did you lint the configuration? Any issues?
>
> (Yes, I do advocate ok_locales usage, were appropriate. It kicks some
> serious butt here.)
>
>
>
>> and we sent email messages to our servers with russian and korean encoding
>>
>> im dbl checkin on what if any of those rules actually were reported
>>
>
> Can you put a raw, sample message up a pastebin or any accessible place
> (don't paste, don't attach to a post), that you believe should be caught
> but wasn't by ok_locales, please?
>
>
>
>> btw that doc u sent doesnt have anything on ok_language
>> it could use some updates on that dont forget to tell people to uncoment
>> the TextCat module there too
>>
>
> Sorry, if I'm wrong, but... That suggests you did *not* lint your
> configuration. You would have gotten a warning when using ok_languages
> without the TextCat plugin enabled.
>
> guenther
>
>
>
Re: koi8-r ok_locales and ok_languages
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
> # Set ok_locals to just en
> ok_locales en
Please note that the comment above is wrong. It actually means
# all Western char sets in general
> # Set ok_languages to just english and french
> ok_languages en fr
>
> # Set inactive_languages all defaults plus russian and korean
> inactive_languages bs cy eo et eu fy ga gd is ko la lt lv rm ru sa sco sl yi
By that, you disabled Russian and Korean detection, err, guessing. ;)
The TextCat plugin thus will never detect any mail written in either
Russian or Korean -- and subsequently will not be able to tell it is an
UNWANTED_LANGUAGE_BODY, because it didn't guess any language.
In other words: TextCat will not identify text to be written in the
languages specified by inactive_languages, because you told it to. If it
can't guess a language with sufficient confidence, it can't be sure it
is outside the list of ok_languages either.
See the previously referenced documentation for the TextCat plugin.
> all three were run seperately and together with no difference, aka spam
> still got through
>
> spamassassin was restarted /etc/init.d/spamassassin restart
With "ok_locales en", did you lint the configuration? Any issues?
(Yes, I do advocate ok_locales usage, were appropriate. It kicks some
serious butt here.)
> and we sent email messages to our servers with russian and korean encoding
>
> im dbl checkin on what if any of those rules actually were reported
Can you put a raw, sample message up a pastebin or any accessible place
(don't paste, don't attach to a post), that you believe should be caught
but wasn't by ok_locales, please?
> btw that doc u sent doesnt have anything on ok_language
> it could use some updates on that dont forget to tell people to uncoment
> the TextCat module there too
Sorry, if I'm wrong, but... That suggests you did *not* lint your
configuration. You would have gotten a warning when using ok_languages
without the TextCat plugin enabled.
guenther
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: koi8-r ok_locales and ok_languages
Posted by Clayton Cottingham <cl...@ardishealth.com>.
all :
yes textcat was loaded as i noticed that from googling for info
here are my edits i placed in local.cf
===============
# Set ok_locals to just en
ok_locales en
# Set ok_languages to just english and
french
ok_languages en
fr
# Set inactive_languages all defaults plus russian and korean
inactive_languages bs cy eo et eu fy ga gd is ko la lt lv rm ru sa sco
sl yi
=========
all three were run seperately and together with no difference, aka spam
still got through
spamassassin was restarted /etc/init.d/spamassassin restart
and we sent email messages to our servers with russian and korean encoding
im dbl checkin on what if any of those rules actually were reported
btw that doc u sent doesnt have anything on ok_language
it could use some updates on that dont forget to tell people to uncoment
the TextCat module there too
Duane Hill wrote:
> On Tue, 21 Oct 2008, Clayton Cottingham wrote:
>
>> hi all
>>
>> trying to figure a way to filter out, or better yet score charset koi8-r
>> down some
>>
>> ok_locales and ok_languages do not work as expected
>> inactive_languages is a no go as well
>>
>> any help appreciated
>
> For ok_languages, do you have the TextCat plugin loaded? In the
> current version of SA, it is located in the v310.pre file where the
> local.cf config lives:
>
> loadplugin Mail::SpamAssassin::Plugin::TextCat
>
> I believe the default setting has the plugin commented out (does not
> load) by default.
>
> See:
> perldoc Mail::SpamAssassin::Plugin::TextCat
> or:
>
> http://spamassassin.apache.org/full/3.2.x/doc/Mail_SpamAssassin_Plugin_TextCat.html
>
> for details.
Re: koi8-r ok_locales and ok_languages
Posted by Duane Hill <d....@yournetplus.com>.
On Tue, 21 Oct 2008, Clayton Cottingham wrote:
> hi all
>
> trying to figure a way to filter out, or better yet score charset koi8-r
> down some
>
> ok_locales and ok_languages do not work as expected
> inactive_languages is a no go as well
>
> any help appreciated
For ok_languages, do you have the TextCat plugin loaded? In the current
version of SA, it is located in the v310.pre file where the local.cf
config lives:
loadplugin Mail::SpamAssassin::Plugin::TextCat
I believe the default setting has the plugin commented out (does not load)
by default.
See:
perldoc Mail::SpamAssassin::Plugin::TextCat
or:
http://spamassassin.apache.org/full/3.2.x/doc/Mail_SpamAssassin_Plugin_TextCat.html
for details.