You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Dave Goodrich <ld...@tls.net> on 2004/11/03 22:40:59 UTC
SA 3.01 scoring very low
Good afternoon,
I just finished testing an upgrade of SA to 3.01 and my scores fell
through the floor. Read the docs, tried to use the Wiki, followed
everyone else's upgrade on the list. Not sure just what went wrong.
DAve
Here is a sample output of spamassassin -D < test_spam (a known spam
that had been caught and scored as follows,
******** previous score ********
Content analysis details: (21.9 hits, 4.0 required)
2.8 SUBJ_VIAGRA Subject includes "viagra"
2.6 LOCAL_OBFU_REGALIS_SUBJ Obfuscated 'REGALIS' in subject
2.6 LOCAL_OBFU_CIALIS_SUBJ Obfuscated 'CIALIS' in subject
0.9 SUBJ_BUY 'Subject' starts with Buy, Buying
1.8 LOCAL_OBFU_CIALIS BODY: Obfuscated 'CIALIS' in body
1.8 LOCAL_OBFU_REGALIS BODY: Obfuscated 'REGALIS' in body
3.0 SPAMCOP_URI_RBL URI's domain appears in spamcop database at
sc.surbl.org
[a1medz.com is blacklisted in URI RBL at]
[multi.surbl.org]
2.1 WS_URI_RBL URI's domain appears in ws database at
ws.surbl.org
[a1medz.com is blacklisted in URI RBL at]
[multi.surbl.org]
2.1 OB_URI_RBL URI's domain appears in ws database at
ob.surbl.org
[a1medz.com is blacklisted in URI RBL at]
[multi.surbl.org]
2.2 SARE_URI_MEDS URI: domain selling meds
********* my local.cf ***********
# Add your own customisations to this file. See 'man
Mail::SpamAssassin::Conf'
# for details of what can be tweaked.
required_hits 5.0
rewrite_header Subject *****SPAM*****
report_safe 1
skip_rbl_checks 1
use_auto_whitelist 0
bayes_auto_learn 0
use_bayes 0
use_pyzor 0
use_dcc 0
use_razor2 0
dns_available no
******** SA 3.01 debug output *********
bash-2.05b# spamassassin -D < test_spam
debug: SpamAssassin version 3.0.1
debug: Score set 0 chosen.
debug: running in taint mode? yes
debug: Running in taint mode, removing unsafe env vars, and resetting PATH
debug: PATH included '/sbin', keeping.
debug: PATH included '/bin', keeping.
debug: PATH included '/usr/sbin', keeping.
debug: PATH included '/usr/bin', keeping.
debug: PATH included '/usr/games', keeping.
debug: PATH included '/usr/local/sbin', keeping.
debug: PATH included '/usr/local/bin', keeping.
debug: Final PATH set to:
/sbin:/bin:/usr/sbin:/usr/bin:/usr/games:/usr/local/sbin:/usr/local/bin:/usr/X11R6/bin
debug: using "/etc/mail/spamassassin/init.pre" for site rules init.pre
debug: config: read file /etc/mail/spamassassin/init.pre
debug: using "/usr/local/share/spamassassin" for default rules dir
debug: config: read file /usr/local/share/spamassassin/10_misc.cf
debug: config: read file /usr/local/share/spamassassin/20_anti_ratware.cf
debug: config: read file /usr/local/share/spamassassin/20_body_tests.cf
debug: config: read file /usr/local/share/spamassassin/20_compensate.cf
debug: config: read file /usr/local/share/spamassassin/20_dnsbl_tests.cf
debug: config: read file /usr/local/share/spamassassin/20_drugs.cf
debug: config: read file /usr/local/share/spamassassin/20_fake_helo_tests.cf
debug: config: read file /usr/local/share/spamassassin/20_head_tests.cf
debug: config: read file /usr/local/share/spamassassin/20_html_tests.cf
debug: config: read file /usr/local/share/spamassassin/20_meta_tests.cf
debug: config: read file /usr/local/share/spamassassin/20_phrases.cf
debug: config: read file /usr/local/share/spamassassin/20_porn.cf
debug: config: read file /usr/local/share/spamassassin/20_ratware.cf
debug: config: read file /usr/local/share/spamassassin/20_uri_tests.cf
debug: config: read file /usr/local/share/spamassassin/23_bayes.cf
debug: config: read file /usr/local/share/spamassassin/25_body_tests_es.cf
debug: config: read file /usr/local/share/spamassassin/25_hashcash.cf
debug: config: read file /usr/local/share/spamassassin/25_spf.cf
debug: config: read file /usr/local/share/spamassassin/25_uribl.cf
debug: config: read file /usr/local/share/spamassassin/30_text_de.cf
debug: config: read file /usr/local/share/spamassassin/30_text_fr.cf
debug: config: read file /usr/local/share/spamassassin/30_text_nl.cf
debug: config: read file /usr/local/share/spamassassin/30_text_pl.cf
debug: config: read file /usr/local/share/spamassassin/50_scores.cf
debug: config: read file /usr/local/share/spamassassin/60_whitelist.cf
debug: using "/etc/mail/spamassassin" for site rules dir
debug: config: read file /etc/mail/spamassassin/70_sare_adult.cf
debug: config: read file /etc/mail/spamassassin/70_sare_bayes_poison_nxm.cf
debug: config: read file /etc/mail/spamassassin/70_sare_genlsubj0.cf
debug: config: read file /etc/mail/spamassassin/70_sare_header0.cf
debug: config: read file /etc/mail/spamassassin/70_sare_html0.cf
debug: config: read file /etc/mail/spamassassin/70_sare_oem.cf
debug: config: read file /etc/mail/spamassassin/70_sare_random.cf
debug: config: read file /etc/mail/spamassassin/70_sare_ratware.cf
debug: config: read file /etc/mail/spamassassin/70_sare_specific.cf
debug: config: read file /etc/mail/spamassassin/70_sare_spoof.cf
debug: config: read file /etc/mail/spamassassin/99_sare_fraud_post25x.cf
debug: config: read file /etc/mail/spamassassin/global_whitelist.cf
debug: config: read file /etc/mail/spamassassin/local.cf
debug: using "/root/.spamassassin" for user state dir
debug: using "/root/.spamassassin/user_prefs" for user prefs file
debug: config: read file /root/.spamassassin/user_prefs
debug: plugin: loading Mail::SpamAssassin::Plugin::URIDNSBL from @INC
debug: plugin: registered
Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x89bf6f4)
debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x89bf6f4)
implements 'parse_config'
debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x89bf6f4)
inhibited further callbacks
debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x89bf6f4)
inhibited further callbacks
debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x89bf6f4)
inhibited further callbacks
debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x89bf6f4)
inhibited further callbacks
debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x89bf6f4)
inhibited further callbacks
debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x89bf6f4)
inhibited further callbacks
debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x89bf6f4)
inhibited further callbacks
debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x89bf6f4)
inhibited further callbacks
debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x89bf6f4)
inhibited further callbacks
debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x89bf6f4)
inhibited further callbacks
debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x89bf6f4)
inhibited further callbacks
debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x89bf6f4)
inhibited further callbacks
debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x89bf6f4)
inhibited further callbacks
debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x89bf6f4)
inhibited further callbacks
debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x89bf6f4)
inhibited further callbacks
debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x89bf6f4)
inhibited further callbacks
debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x89bf6f4)
inhibited further callbacks
debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x89bf6f4)
inhibited further callbacks
debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x89bf6f4)
inhibited further callbacks
debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x89bf6f4)
inhibited further callbacks
debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x89bf6f4)
inhibited further callbacks
debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x89bf6f4)
inhibited further callbacks
debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x89bf6f4)
inhibited further callbacks
debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x89bf6f4)
inhibited further callbacks
debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x89bf6f4)
inhibited further callbacks
debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x89bf6f4)
inhibited further callbacks
debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x89bf6f4)
inhibited further callbacks
debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x89bf6f4)
inhibited further callbacks
debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x89bf6f4)
inhibited further callbacks
debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x89bf6f4)
inhibited further callbacks
debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x89bf6f4)
inhibited further callbacks
debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x89bf6f4)
inhibited further callbacks
debug: Score set 1 chosen.
debug: received-header: unknown format:
debug: received-header: unknown format:
debug: received-header: unknown format:
debug: received-header: unknown format:
debug: metadata: X-Spam-Relays-Trusted:
debug: metadata: X-Spam-Relays-Untrusted:
debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x89bf6f4)
implements 'parsed_metadata'
debug: dns_available set to no in config file, skipping test
debug: ---- MIME PARSER START ----
debug: main message type: text/plain
debug: parsing normal part
debug: added part, type: text/plain
debug: ---- MIME PARSER END ----
debug: decoding: no encoding detected
debug: uri found: http://a1medz.com/sup/
debug: uri found: http://a1medz.com/rr.php
debug: Running tests for priority: 0
debug: running header regexp tests; score so far=0
debug: time cannot be parsed:
debug: all '*To' addrs:
debug: all '*From' addrs:
debug: running body-text per-line regexp tests; score so far=-0.962
debug: running uri tests; score so far=-0.962
debug: registering glue method for check_uridnsbl
(Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x89bf6f4))
debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x89bf6f4)
implements 'check_tick'
debug: running raw-body-text per-line regexp tests; score so far=-0.962
debug: running full-text regexp tests; score so far=-0.962
debug: DCCifd is not available: no r/w dccifd socket found.
debug: Running tests for priority: 500
debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x89bf6f4)
implements 'check_post_dnsbl'
debug: running meta tests; score so far=-0.962
debug: running header regexp tests; score so far=0.634
debug: running body-text per-line regexp tests; score so far=0.634
debug: running uri tests; score so far=0.634
debug: running raw-body-text per-line regexp tests; score so far=0.634
debug: running full-text regexp tests; score so far=0.634
debug: Running tests for priority: 1000
debug: running meta tests; score so far=0.634
debug: running header regexp tests; score so far=0.634
debug: running body-text per-line regexp tests; score so far=0.634
debug: running uri tests; score so far=0.634
debug: running raw-body-text per-line regexp tests; score so far=0.634
debug: running full-text regexp tests; score so far=0.634
debug: is spam? score=0.634 required=5
debug:
tests=ALL_TRUSTED,DRUGS_ERECTILE,FROM_NO_LOWER,INVALID_DATE,MISSING_SUBJECT,RM_hm_EmtyMsgid
debug:
subtests=__DRUGS_ERECTILE3,__DRUGS_ERECTILE_C,__SARE_HTML_HAS_MSG,__UNUSABLE_MSGID
Subject:
Buy Regalis, also known as Superviagra or Cialis
From:
"Jefferson Jennings" <je...@bank-banque-canada.ca>
Date:
Wed, 03 Nov 2004 11:58:37 -0400
To:
ldg@tls.net
Received:
(qmail 1168 invoked from network); 3 Nov 2004 15:58:13 -0000
Received:
from unknown (HELO avhost.tls.net) (10.0.241.141) by 0 with SMTP; 3 Nov
2004 15:58:13 -0000
Received:
from wtnet.ch (219-84-81-213-adsl-tpe.static.so-net.net.tw
[219.84.81.213]) by avhost.tls.net (8.12.8p1/8.12.8) with SMTP id
iA3Fvw2Q080054 for <ld...@tls.net>; Wed, 3 Nov 2004 10:58:08 -0500 (EST)
(envelope-from jeffersonjennings_aq@bank-banque-canada.ca)
Received:
from 72.114.1.201 by smtp.bank-banque-canada.ca; Wed, 03 Nov 2004
15:58:42 +0000
Message-ID:
<33...@wtnet.ch>
MIME-Version:
1.0
Content-Type:
text/plain; charset="iso-8859-1"
Content-Transfer-Encoding:
8bit
X-TLS.net-Anti-Virus-Scanner-Information:
Please contact support@tls.net for more information
X-TLS.net-Anti-Virus-Scanner:
Found to be clean
X-MailScanner-From:
X-Spam-Checker-Version: SpamAssassin 3.0.1 (2004-10-22) on avhost.tls.net
X-Spam-Status: No, score=0.6 required=5.0 tests=ALL_TRUSTED,DRUGS_ERECTILE,
FROM_NO_LOWER,INVALID_DATE,MISSING_SUBJECT,RM_hm_EmtyMsgid
autolearn=disabled version=3.0.1
X-Spam-Level:
--
Systems Administrator
http://www.tls.net
Get rid of Unwanted Emails...get TLS Spam Blocker!
Re: SA 3.01 scoring very low
Posted by Dave Goodrich <ld...@tls.net>.
Thanks everyone, testing with several messages and comparing to 2.64
scores looks good now.
Three issues,
1) My test message was munged and SA had problems parsing the headers.
Used unmangled messages and SA parsed them fine.
2) Set trusted networks to 127.0.0.1, so no network is trusted.
3) set "dns_available yes", this stopped the testing of dns
availability, while still allowing dns tests themselves to run.
Of note, setting "skip_rbl_checks 1" does not stop SURBL tests, which is
good. Just stops the rbl checks for smtp connections.
DAve
Matt Kettler wrote:
> At 09:54 AM 11/4/2004 -0500, Dave Goodrich wrote:
>
>>> Yes I just submitted a bug on the matter.. Currently ALL_TRUSTED
>>> fires whenever there are no untrusted relays detected.. However, it
>>> fails to check that any trusted relays exist...
>>> I opened this bug to suggest a fix for ALL_TRUSTED:
>>> http://bugzilla.spamassassin.org/show_bug.cgi?id=3949
>>> However, the Received: path parsing bug is something I leave up to
>>> Dave to file.
>>
>> No need, I rechecked my test message and it had some formatting
>> problems from being transfered off my workstation (Thunderbird) and
>> onto the SA box. I grabbed a couple other messages right out of the
>> Maildir and they parsed fine.
>>
>> I believe the issue with the headers was of my making, not a SA problem
>
>
> Fair enough, thanks for the follow-up.
>
> I still think it's worth fixing ALL_TRUSTED just in case.
>
> There's at least one valid open bug regarding Received: formats..
>
> http://bugzilla.spamassassin.org/show_bug.cgi?id=3600
>
> And many others are possible, so it's definitely worth the preventative
> measures.
>
>
--
Systems Administrator
http://www.tls.net
Get rid of Unwanted Emails...get TLS Spam Blocker!
Re: SA 3.01 scoring very low
Posted by Matt Kettler <mk...@comcast.net>.
At 09:54 AM 11/4/2004 -0500, Dave Goodrich wrote:
>>Yes I just submitted a bug on the matter.. Currently ALL_TRUSTED fires
>>whenever there are no untrusted relays detected.. However, it fails to
>>check that any trusted relays exist...
>>I opened this bug to suggest a fix for ALL_TRUSTED:
>> http://bugzilla.spamassassin.org/show_bug.cgi?id=3949
>>However, the Received: path parsing bug is something I leave up to Dave
>>to file.
>No need, I rechecked my test message and it had some formatting problems
>from being transfered off my workstation (Thunderbird) and onto the SA
>box. I grabbed a couple other messages right out of the Maildir and they
>parsed fine.
>
>I believe the issue with the headers was of my making, not a SA problem
Fair enough, thanks for the follow-up.
I still think it's worth fixing ALL_TRUSTED just in case.
There's at least one valid open bug regarding Received: formats..
http://bugzilla.spamassassin.org/show_bug.cgi?id=3600
And many others are possible, so it's definitely worth the preventative
measures.
Re: SA 3.01 scoring very low
Posted by Dave Goodrich <ld...@tls.net>.
Matt Kettler wrote:
> At 02:19 PM 11/4/2004 +0000, Sean Doherty wrote:
>
>> Matt, does this mean that even if trusted_networks is set in local.cf,
>> SpamAssassin will fire the ALL_TRUSTED rule even if it can't parse
>> the received headers? i.e. Since there are no parsable received
>> headers, SA will assume that all must have been trusted?
>
>
> Yes I just submitted a bug on the matter.. Currently ALL_TRUSTED fires
> whenever there are no untrusted relays detected.. However, it fails to
> check that any trusted relays exist...
>
> I opened this bug to suggest a fix for ALL_TRUSTED:
>
> http://bugzilla.spamassassin.org/show_bug.cgi?id=3949
>
> However, the Received: path parsing bug is something I leave up to Dave
> to file.
No need, I rechecked my test message and it had some formatting problems
from being transfered off my workstation (Thunderbird) and onto the SA
box. I grabbed a couple other messages right out of the Maildir and they
parsed fine.
I believe the issue with the headers was of my making, not a SA problem.
DAve
--
Systems Administrator
http://www.tls.net
Get rid of Unwanted Emails...get TLS Spam Blocker!
Re: SA 3.01 scoring very low
Posted by Matt Kettler <mk...@comcast.net>.
At 02:19 PM 11/4/2004 +0000, Sean Doherty wrote:
>Matt, does this mean that even if trusted_networks is set in local.cf,
>SpamAssassin will fire the ALL_TRUSTED rule even if it can't parse
>the received headers? i.e. Since there are no parsable received
>headers, SA will assume that all must have been trusted?
Yes I just submitted a bug on the matter.. Currently ALL_TRUSTED fires
whenever there are no untrusted relays detected.. However, it fails to
check that any trusted relays exist...
I opened this bug to suggest a fix for ALL_TRUSTED:
http://bugzilla.spamassassin.org/show_bug.cgi?id=3949
However, the Received: path parsing bug is something I leave up to Dave to
file.
Really mis-parsed Received: headers is a serious bug, the fix to
ALL_TRUSTED is just damage control.
Re: {SPAM} SA 3.01 scoring very low
Posted by Sean Doherty <se...@copperfasten.com>.
On Wed, 2004-11-03 at 21:52, Matt Kettler wrote:
> At 04:40 PM 11/3/2004, Dave Goodrich wrote:
> >Good afternoon,
> >
> >I just finished testing an upgrade of SA to 3.01 and my scores fell
> >through the floor. Read the docs, tried to use the Wiki, followed everyone
> >else's upgrade on the list. Not sure just what went wrong.
> >
> >DAve
> >
> >Here is a sample output of spamassassin -D < test_spam (a known spam that
> >had been caught and scored as follows,
>
> <snip>
> >debug: received-header: unknown format:
> >debug: received-header: unknown format:
> >debug: received-header: unknown format:
> >debug: received-header: unknown format:
>
> <snip>
>
> There's the cause of your problem.. SA is having problems parsing your
> received headers.
>
> As a result, SA is failing to properly detect a trust path, and is
> triggering ALL_TRUSTED, which should never happen for outside mail.
> In the short term, force ALL_TRUSTED to 0
Matt, does this mean that even if trusted_networks is set in local.cf,
SpamAssassin will fire the ALL_TRUSTED rule even if it can't parse
the received headers? i.e. Since there are no parsable received
headers, SA will assume that all must have been trusted?
Seems a bit aggressive to me...
- Sean
Re: {SPAM} SA 3.01 scoring very low
Posted by Matt Kettler <mk...@evi-inc.com>.
At 04:40 PM 11/3/2004, Dave Goodrich wrote:
>Good afternoon,
>
>I just finished testing an upgrade of SA to 3.01 and my scores fell
>through the floor. Read the docs, tried to use the Wiki, followed everyone
>else's upgrade on the list. Not sure just what went wrong.
>
>DAve
>
>Here is a sample output of spamassassin -D < test_spam (a known spam that
>had been caught and scored as follows,
<snip>
>debug: received-header: unknown format:
>debug: received-header: unknown format:
>debug: received-header: unknown format:
>debug: received-header: unknown format:
<snip>
There's the cause of your problem.. SA is having problems parsing your
received headers.
As a result, SA is failing to properly detect a trust path, and is
triggering ALL_TRUSTED, which should never happen for outside mail.
>debug:
>tests=ALL_TRUSTED,DRUGS_ERECTILE,FROM_NO_LOWER,INVALID_DATE,MISSING_SUBJECT,RM_hm_EmtyMsgid
>debug:
>subtests=__DRUGS_ERECTILE3,__DRUGS_ERECTILE_C,__SARE_HTML_HAS_MSG,__UNUSABLE_MSGID
In the short term, force ALL_TRUSTED to 0
score ALL_TRUSTED 0
Open a bug report pointing out that SA isn't parsing your recieved headers.
After you create the bug, attach a sample message to the bug so that the
devs can test and fix things.
Re: SA 3.01 scoring very low
Posted by Dave Goodrich <ld...@tls.net>.
Matt Kettler wrote:
> At 10:17 AM 11/4/2004, Sean Doherty wrote:
>
>> > JMHO, but shouldn't all networks be considered untrusted unless a user
>> > specifies otherwise?
>>
>> I got to agree with you there - especially given that the inference
>> algorithm doesn't work in every environment.
>
>
> Unfortunately this only solves one aspect of the problem.
>
> SA NEEDS to have the correct trust path.
>
> Trusting nobody is just as bad as trusting everyone. Trusting nobody
> breaks whitelist_from_rcvd, for example.
This is all becoming very confusing about what effect the trusted
networks code has on the rest of SA. Possibly I have not read the conf
pages correctly.
"internal_networks ip.add.re.ss[/mask] ... (default: none)
If neither trusted_networks or internal_networks is set, no
addresses will be considered local; in other words, any relays past the
machine where SpamAssassin is running will be considered external."
And trusted?
"whitelist_from_rcvd addr@lists.sourceforge.net sourceforge.net
Note that this requires that internal_networks be correct. For
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
simple cases, it will be, but for a complex network, or running with DNS
checks off or with -L, you may get better results by setting that
parameter."
I'm confused here, if I set no trust params, then all networks are
trusted by default. But if I trust no networks, then I cannot use
whitelist_from_rcvd to define a trusted relay?
To me that says, in order to define a trusted relay via
whitelist_from_rcvd, I first must trust ALL relays, or put all the
relays I have in whitelist_from_rcvd into my trusted networks as well.
DAve
--
Systems Administrator
http://www.tls.net
Get rid of Unwanted Emails...get TLS Spam Blocker!
Re: SA 3.01 scoring very low
Posted by Matt Kettler <mk...@evi-inc.com>.
At 11:14 AM 11/4/2004, Jim Maul wrote:
>While i agree that trusting no one doesnt really solve the problem, I dont
>believe it is "just as bad" as trusting everyone. Trusting
>everyone stops other rules from firing and adds atleast -2.something to
>every message. This seems far worse than trusting no one and breaking
>whitelist_from_rcvd
While I'll concede it may not be "just as bad" it's still much worse than
you think.
LOTS of rules in SA depend on trust. Not just whitelist_from_rcvd and
ALL_TRUSTED.
All of these rules are broken by a broken trust path, some in ways that
cause FPs, others just missing out on score:
HELO_DYNAMIC_*
FAKE_HELO_MAIL_COM_DOM
RCVD_IN_BSP_*
MSGID_FROM_MTA_ID
FORGED_RCVD_*
AWL
trust plays into "notfirsthop" as well, so all these DNSBLs get broken:
RCVD_IN_NJABL_DUL
RCVD_IN_SORBS_DUL
RCVD_IN_XBL
RCVD_IN_DSBL
RCVD_IN_MAPS_DUL
Re: SA 3.01 scoring very low
Posted by Jim Maul <jm...@elih.org>.
Matt Kettler wrote:
> At 10:17 AM 11/4/2004, Sean Doherty wrote:
>
>> > JMHO, but shouldn't all networks be considered untrusted unless a user
>> > specifies otherwise?
>>
>> I got to agree with you there - especially given that the inference
>> algorithm doesn't work in every environment.
>
>
> Unfortunately this only solves one aspect of the problem.
>
> SA NEEDS to have the correct trust path.
>
> Trusting nobody is just as bad as trusting everyone. Trusting nobody
> breaks whitelist_from_rcvd, for example.
>
>
While i agree that trusting no one doesnt really solve the problem, I
dont believe it is "just as bad" as trusting everyone. Trusting
everyone stops other rules from firing and adds atleast -2.something to
every message. This seems far worse than trusting no one and breaking
whitelist_from_rcvd.
-Jim
Re: SA 3.01 scoring very low
Posted by Matt Kettler <mk...@evi-inc.com>.
At 10:17 AM 11/4/2004, Sean Doherty wrote:
> > JMHO, but shouldn't all networks be considered untrusted unless a user
> > specifies otherwise?
>
>I got to agree with you there - especially given that the inference
>algorithm doesn't work in every environment.
Unfortunately this only solves one aspect of the problem.
SA NEEDS to have the correct trust path.
Trusting nobody is just as bad as trusting everyone. Trusting nobody
breaks whitelist_from_rcvd, for example.
Re: SA 3.01 scoring very low
Posted by Sean Doherty <se...@copperfasten.com>.
On Thu, 2004-11-04 at 15:04, Dave Goodrich wrote:
> > Check out trusted_network section of Mail::SpamAssassin::Conf
> > i.e no RBL tests on trusted networks.
> "If you're running with DNS checks enabled, SpamAssassin includes code
> to infer your trusted networks on the fly, so this may not be necessary.
> (Thanks to Scott Banister and Andrew Flury for the inspiration for this
> algorithm.) This inference works as follows:"
>
> This seems backwards to me. If a user does nothing, then his network
> will be considered trusted by default? We are an ISP, and SA is running
> on our toasters. I don't want any machine trusted as that leaves a door
> open for my smtp relay users (viruses, trojans, just bad folks) to spam
> local users.
>
> JMHO, but shouldn't all networks be considered untrusted unless a user
> specifies otherwise?
I got to agree with you there - especially given that the inference
algorithm doesn't work in every environment.
- Sean
Re: SA 3.01 scoring very low
Posted by Dave Goodrich <ld...@tls.net>.
Sean Doherty wrote:
> On Thu, 2004-11-04 at 14:14, Dave Goodrich wrote:
>
>>Sean Doherty wrote:
>>
>>I will look into that, I didn't set it as I want no network to be
>>trusted. I'll reread what I can find on that.
>
> Just set trusted_network 127.0.0.1
Yes, this fixed it.
>
>>>Since you hit ALL_TRUSTED certain other DNS based tests are not
>>>run.
>>
>>Eh? Where do I find this out?
>
>
> Check out trusted_network section of Mail::SpamAssassin::Conf
> i.e no RBL tests on trusted networks.
"If you're running with DNS checks enabled, SpamAssassin includes code
to infer your trusted networks on the fly, so this may not be necessary.
(Thanks to Scott Banister and Andrew Flury for the inspiration for this
algorithm.) This inference works as follows:"
This seems backwards to me. If a user does nothing, then his network
will be considered trusted by default? We are an ISP, and SA is running
on our toasters. I don't want any machine trusted as that leaves a door
open for my smtp relay users (viruses, trojans, just bad folks) to spam
local users.
JMHO, but shouldn't all networks be considered untrusted unless a user
specifies otherwise?
DAve
--
Systems Administrator
http://www.tls.net
Get rid of Unwanted Emails...get TLS Spam Blocker!
Re: SA 3.01 scoring very low
Posted by Sean Doherty <se...@copperfasten.com>.
On Thu, 2004-11-04 at 14:14, Dave Goodrich wrote:
> Sean Doherty wrote:
> > On Wed, 2004-11-03 at 21:40, Dave Goodrich wrote:
> >
> >>Good afternoon,
> >>
> >>I just finished testing an upgrade of SA to 3.01 and my scores fell
> >>through the floor. Read the docs, tried to use the Wiki, followed
> >>everyone else's upgrade on the list. Not sure just what went wrong.
> >
> >
> >>X-Spam-Checker-Version: SpamAssassin 3.0.1 (2004-10-22) on avhost.tls.net
> >>X-Spam-Status: No, score=0.6 required=5.0 tests=ALL_TRUSTED,DRUGS_ERECTILE,
> >> FROM_NO_LOWER,INVALID_DATE,MISSING_SUBJECT,RM_hm_EmtyMsgid
> >> autolearn=disabled version=3.0.1
> >
> >
> > You need to specify trusted_networks in local.cf, otherwise
> > you're going to continue to hit the ALL_TRUSTED rule which can
> > *decrease* your score by up to -3.3. If you don't specify
> > trusted_networks then SpamAssassin infers what your trusted
> > networks are - and the inference algorithm may not always get
> > the correct result. For instance if your mail relay/server is
> > on a private network and NATed thru a firewall, then the
> > algorithm may infer incorrectly that the connecting mail server
> > is trusted. i.e. the algorithm assumes that since you're a
> > private address, then the next hop server must belong to you
> > since your MX must be public. However it does not take NAT
> > into account. Setting trusted_networks appropriately will solve
> > this issue (I don't think SA 2.64 has the ALL_TRUSTED rule - or
> > at least it scores low).
> I will look into that, I didn't set it as I want no network to be
> trusted. I'll reread what I can find on that.
Just set trusted_network 127.0.0.1
> >
> > Since you hit ALL_TRUSTED certain other DNS based tests are not
> > run.
> Eh? Where do I find this out?
Check out trusted_network section of Mail::SpamAssassin::Conf
i.e no RBL tests on trusted networks.
> I don't want any networks trusted, infered or otherwise. So I left
> trusted_networks and internal_networks both blank.
My understanding is that if unset trusted_networks will be infered.
Setting it to the loopback address and/or the host IP address will
prevent this.
> > Also skip_rbl_checks will do just that.
> Umm I don't follow you there, are you saying skip_rbl_checks will skip
> SURBL? Because if it does, I'll need to go back to 2.64.
No. Just pointing out that no RBL tests will not be run.
Also, Matt Kettler pointed out in this thread that reason for the
ALL_TRUSTED firing may not be entirely related invalid inference
of trust, but because the Received headers had unknown format in
the debug output.
- Sean
Re: SA 3.01 scoring very low
Posted by Dave Goodrich <ld...@tls.net>.
Sean Doherty wrote:
> On Wed, 2004-11-03 at 21:40, Dave Goodrich wrote:
>
>>Good afternoon,
>>
>>I just finished testing an upgrade of SA to 3.01 and my scores fell
>>through the floor. Read the docs, tried to use the Wiki, followed
>>everyone else's upgrade on the list. Not sure just what went wrong.
>
>
>>X-Spam-Checker-Version: SpamAssassin 3.0.1 (2004-10-22) on avhost.tls.net
>>X-Spam-Status: No, score=0.6 required=5.0 tests=ALL_TRUSTED,DRUGS_ERECTILE,
>> FROM_NO_LOWER,INVALID_DATE,MISSING_SUBJECT,RM_hm_EmtyMsgid
>> autolearn=disabled version=3.0.1
>
>
> You need to specify trusted_networks in local.cf, otherwise
> you're going to continue to hit the ALL_TRUSTED rule which can
> *decrease* your score by up to -3.3. If you don't specify
> trusted_networks then SpamAssassin infers what your trusted
> networks are - and the inference algorithm may not always get
> the correct result. For instance if your mail relay/server is
> on a private network and NATed thru a firewall, then the
> algorithm may infer incorrectly that the connecting mail server
> is trusted. i.e. the algorithm assumes that since you're a
> private address, then the next hop server must belong to you
> since your MX must be public. However it does not take NAT
> into account. Setting trusted_networks appropriately will solve
> this issue (I don't think SA 2.64 has the ALL_TRUSTED rule - or
> at least it scores low).
I will look into that, I didn't set it as I want no network to be
trusted. I'll reread what I can find on that.
>
> Since you hit ALL_TRUSTED certain other DNS based tests are not
> run.
Eh? Where do I find this out?
>
> Also is dns unavailable (dns_available no)? This may explain
> why you're not getting SURBL hits (which you should if dns
> is fully operational).
>
I marked DNS unavailable as I don't want the DNS check, I do want DNS
tests run, but only SURBL. Rereading it I think it was too late in the
evening, I need to set "dns_available yes" to stop the dns testing, but
still allow dns tests to run.
My choice for leaving trusted_networks blank was this;
" If trusted_networks is not set and internal_networks is, the value
of internal_networks will be used for this parameter.
If you're running with DNS checks enabled, SpamAssassin includes
code to infer your trusted networks on the fly, so this may not be
necessary."
I don't want any networks trusted, infered or otherwise. So I left
trusted_networks and internal_networks both blank.
> Also skip_rbl_checks will do just that.
Umm I don't follow you there, are you saying skip_rbl_checks will skip
SURBL? Because if it does, I'll need to go back to 2.64.
"By default, SpamAssassin will run RBL checks. If your ISP already does
this for you, set this to 1."
Thanks,
DAve
--
Systems Administrator
http://www.tls.net
Get rid of Unwanted Emails...get TLS Spam Blocker!
Re: SA 3.01 scoring very low
Posted by Sean Doherty <se...@copperfasten.com>.
On Wed, 2004-11-03 at 21:40, Dave Goodrich wrote:
> Good afternoon,
>
> I just finished testing an upgrade of SA to 3.01 and my scores fell
> through the floor. Read the docs, tried to use the Wiki, followed
> everyone else's upgrade on the list. Not sure just what went wrong.
> X-Spam-Checker-Version: SpamAssassin 3.0.1 (2004-10-22) on avhost.tls.net
> X-Spam-Status: No, score=0.6 required=5.0 tests=ALL_TRUSTED,DRUGS_ERECTILE,
> FROM_NO_LOWER,INVALID_DATE,MISSING_SUBJECT,RM_hm_EmtyMsgid
> autolearn=disabled version=3.0.1
You need to specify trusted_networks in local.cf, otherwise
you're going to continue to hit the ALL_TRUSTED rule which can
*decrease* your score by up to -3.3. If you don't specify
trusted_networks then SpamAssassin infers what your trusted
networks are - and the inference algorithm may not always get
the correct result. For instance if your mail relay/server is
on a private network and NATed thru a firewall, then the
algorithm may infer incorrectly that the connecting mail server
is trusted. i.e. the algorithm assumes that since you're a
private address, then the next hop server must belong to you
since your MX must be public. However it does not take NAT
into account. Setting trusted_networks appropriately will solve
this issue (I don't think SA 2.64 has the ALL_TRUSTED rule - or
at least it scores low).
Since you hit ALL_TRUSTED certain other DNS based tests are not
run.
Also is dns unavailable (dns_available no)? This may explain
why you're not getting SURBL hits (which you should if dns
is fully operational). Also skip_rbl_checks will do just that.
Regards,
- Sean