You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spamassassin.apache.org by bu...@bugzilla.spamassassin.org on 2005/12/10 00:01:04 UTC
[Bug 4728] New: DUL rules should use -firsttrusted not -notfirsthop
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4728
Summary: DUL rules should use -firsttrusted not -notfirsthop
Product: Spamassassin
Version: SVN Trunk (Latest Devel Version)
Platform: Other
OS/Version: other
Status: NEW
Severity: normal
Priority: P5
Component: Rules
AssignedTo: dev@spamassassin.apache.org
ReportedBy: spamassassin@dostech.ca
http://article.gmane.org/gmane.mail.spam.spamassassin.general/75005
On 08/12/2005 3:52 PM, Matt Kettler wrote:
> Daryl C. W. O'Shea wrote:
>>That's not what the rule is looking for (the last hop).
>>
>>The rule will lookup any hop that is NOT the FIRST hop. Since the mail
>>first passes through a proxy (the hop we don't check as long as there
>>are other external hops) and then passes through another hop (that we do
>>check) the rule is firing since that second hop is listed.
>>
>
>
> Comment for correctness:
>
> Technically, the "notfirsthop" is a misnomer, and a carry over from really old
> versions of spamassassin. In really old versions, it really worked this way. SA
> Checked every IP except the first hop.
>
> However, SA stopped doing that a long time ago. The implementation of this in SA
> 2.60 and higher is actually "first untrusted host delivering mail to a trusted
> host". The "notfirsthop" name remains in the rules, but it's an artifact, and is
> not really implemented this way.
3.x reverted to the old way. Try it out.
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
[Bug 4728] fix 20_dnsbl_tests.cf to use -lastexternal instead of -notfirsthop
Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4728
jm@jmason.org changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|REOPENED |RESOLVED
Resolution| |FIXED
------- Additional Comments From jm@jmason.org 2007-01-17 07:12 -------
ok, applied.
: jm 1487...; svn commit -m "bug 4728: fix -notfirsthop DNSBL lookup rules to
use -lastexternal instead, since it reduces FPs and is easier for legit senders
to avoid" rules/20_dnsbl_tests.cf rulesrc/sandbox/dos/70_bugs.cf
Sending rulesrc/sandbox/dos/70_bugs.cf
Sending rules/20_dnsbl_tests.cf
Transmitting file data ..
Committed revision 497041.
one (IMO minor) question -- is it wise to leave those reusing the hits from the
-notfirsthop versions?
We have a choice between (a) saying "no", and re-scanning all those messages,
possibly resulting in FPs due to ISPs' changed layouts in the intervening time,
or (b) saying "yes", and accepting the FPs as documented in this bug.
personally, I'd go for (b)... less likelihood of invalid data in the mass-check
results.
------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.
[Bug 4728] [review] DUL rules should only use the last external IP, not all but the first of the external IPs
Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4728
Bug 4728 depends on bug 5294, which changed state.
Bug 5294 Summary: RCVD_IN_XBL should use lastexternal
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5294
What |Old Value |New Value
----------------------------------------------------------------------------
Status|NEW |RESOLVED
Resolution| |FIXED
------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.
[Bug 4728] [review] DUL rules should only use the last external IP, not all but the first of the external IPs
Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4728
------- Additional Comments From jm@jmason.org 2006-01-24 07:23 -------
+1
------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.
[Bug 4728] DUL rules should use -firsttrusted not -notfirsthop
Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4728
------- Additional Comments From spamassassin@dostech.ca 2005-12-19 04:38 -------
-notfirsthop
17.8083 0.1765 0.990 0.79 0.00 RCVD_IN_DSBL
15.3809 0.6409 0.960 0.65 0.00 RCVD_IN_NJABL_DUL
19.1217 0.6115 0.969 0.66 0.00 RCVD_IN_SORBS_DUL
14.2747 0.0691 0.995 0.85 0.00 RCVD_IN_WHOIS_INVALID
21.7952 0.0064 1.000 0.97 0.00 RCVD_IN_XBL
-lastexternal
15.2286 0.1740 0.989 0.78 0.00 T_E_RCVD_IN_DSBL
11.8689 0.5424 0.956 0.65 0.00 T_E_RCVD_IN_NJABL_DUL
18.3798 0.5412 0.971 0.68 0.00 T_E_RCVD_IN_SORBS_DUL
9.3638 0.0217 0.998 0.90 0.00 T_E_RCVD_IN_WHOIS_INVALID
12.6336 0.0192 0.998 0.92 0.00 T_E_RCVD_IN_XBL
-firsttrusted
15.6576 0.1855 0.988 0.77 0.00 T_RCVD_IN_DSBL
12.7466 1.1322 0.918 0.60 0.00 T_RCVD_IN_NJABL_DUL
19.6131 1.2230 0.941 0.61 0.00 T_RCVD_IN_SORBS_DUL
9.5122 0.0755 0.992 0.82 0.00 T_RCVD_IN_WHOIS_INVALID
12.8561 0.0192 0.999 0.92 0.00 T_RCVD_IN_XBL
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
[Bug 4728] [review] DUL rules should only use the last external IP, not all but the first of the external IPs
Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4728
spamassassin@dostech.ca changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|ASSIGNED |RESOLVED
Resolution| |FIXED
------- Additional Comments From spamassassin@dostech.ca 2006-02-03 00:20 -------
3.1:
Sending lib/Mail/SpamAssassin/EvalTests.pm
Sending rules/20_dnsbl_tests.cf
Transmitting file data ..
Committed revision 374540.
------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.
[Bug 4728] DUL rules should use -firsttrusted not -notfirsthop
Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4728
------- Additional Comments From spamassassin@dostech.ca 2005-12-12 18:56 -------
Revision 356300 adds "-lastexternal" which lists (only) the external host
connecting to your internal network (well, actually, the last external host with
a public IP).
Revision 356302 adds rules to my sandbox to test it out.
---
This week's mass-checks against the test rules I added that use "-firsttrusted"
didn't make a whole hell of a lot of sense. Either people are using configs
that utilize trusted_networks that expand beyond their internal_networks (which
the above rules will detect correctly) or timeouts are skewing the results (I
had to increase my rbl_timeout to 60 to get consistent sensible results).
I guess we'll see how these new rules work out next week.
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
[Bug 4728] DUL rules should use -firsttrusted not -notfirsthop
Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4728
spamassassin@dostech.ca changed:
What |Removed |Added
----------------------------------------------------------------------------
AssignedTo|dev@spamassassin.apache.org |spamassassin@dostech.ca
Status|NEW |ASSIGNED
------- Additional Comments From spamassassin@dostech.ca 2006-01-24 04:20 -------
Created an attachment (id=3339)
--> (http://issues.apache.org/SpamAssassin/attachment.cgi?id=3339&action=view)
patch to implement foo-lastexternal
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
[Bug 4728] [review] DUL rules should only use the last external IP, not all but the first of the external IPs
Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4728
jm@jmason.org changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|RESOLVED |REOPENED
Resolution|FIXED |
------- Additional Comments From jm@jmason.org 2006-02-11 03:39 -------
I've just realised, I think the documentation for -notfirsthop, -firsttrusted,
-untrusted qualifiers in lib/Mail/SpamAssassin/Conf.pm is now out of date; it at
least needs to be updated with doco on "-lastexternal".
------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.
[Bug 4728] DUL rules should use -firsttrusted not -notfirsthop
Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4728
------- Additional Comments From spamassassin@dostech.ca 2005-12-24 21:35 -------
20051224-r358194:
-notfirsthop
17.7679 0.3612 0.980 0.78 0.00 RCVD_IN_DSBL
14.8737 1.1648 0.927 0.65 0.00 RCVD_IN_NJABL_DUL
18.3653 1.1308 0.942 0.66 0.00 RCVD_IN_SORBS_DUL
14.6567 0.0838 0.994 0.88 0.00 RCVD_IN_WHOIS_INVALID
20.9553 0.0131 0.999 0.97 0.00 RCVD_IN_XBL
-lastexternal
16.2206 0.3560 0.979 0.77 0.00 T_E_RCVD_IN_DSBL
12.2548 1.1020 0.917 0.64 0.00 T_E_RCVD_IN_NJABL_DUL
18.6292 1.0863 0.945 0.67 0.00 T_E_RCVD_IN_SORBS_DUL
10.3617 0.0419 0.996 0.90 0.00 T_E_RCVD_IN_WHOIS_INVALID
16.5123 0.0419 0.997 0.93 0.00 T_E_RCVD_IN_XBL
-firsttrusted
16.7891 0.3795 0.978 0.77 0.00 T_RCVD_IN_DSBL
13.4486 2.3060 0.854 0.59 0.00 T_RCVD_IN_NJABL_DUL
20.2744 2.4762 0.891 0.60 0.00 T_RCVD_IN_SORBS_DUL
10.5565 0.1492 0.986 0.82 0.00 T_RCVD_IN_WHOIS_INVALID
16.8329 0.0471 0.997 0.93 0.00 T_RCVD_IN_XBL
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
[Bug 4728] DUL rules should use -firsttrusted not -notfirsthop
Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4728
------- Additional Comments From spamassassin@dostech.ca 2005-12-12 21:46 -------
I had tried 30 seconds but the dynamic timeout was still timing out before I got
back the SORBS DUL hits I was looking for. I didn't try anything between that
and 60.
Processing finishes in about 3.5 seconds with all of the dns lookups finishing.
In any case, the 15 second default for the rbl_timeout was inflicting way too
much variability in my tests.
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
[Bug 4728] [review] DUL rules should only use the last external IP, not all but the first of the external IPs
Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4728
------- Additional Comments From vectro@vectro.org 2006-11-27 08:45 -------
(In reply to comment #1)
> "notfirsthop" (as currently done in 3.1.0) breaks for this (somewhat rare, but
> legitimate) case:
> public IP -> public (dyn) -> ISP -> Recipient MX -> SA.
The problem is that it's impossible to detect if this case is legitimate or not.
There's nothing to prevent a spammer from adding a forged Recieved address that
shows the message origin to be something else entirely.
> "firstuntrusted" breaks for the case that a user decides to trust an outside
> ISPs MX to be non-forging.
This IMO is a more serious concern.
See also bug 5209, which is a somewhat related issue.
------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.
[Bug 4728] [review] DUL rules should only use the last external IP, not all but the first of the external IPs
Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4728
jm@jmason.org changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|REOPENED |RESOLVED
Resolution| |FIXED
------- Additional Comments From jm@jmason.org 2006-02-23 23:17 -------
r380240: trivial doc fix.
------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.
[Bug 4728] fix 20_dnsbl_tests.cf to use -lastexternal instead of -notfirsthop
Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4728
jm@jmason.org changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|RESOLVED |REOPENED
Resolution|FIXED |
Summary|[review] DUL rules should |fix 20_dnsbl_tests.cf to use
|only use the last external |-lastexternal instead of -
|IP, not all but the first of|notfirsthop
|the external IPs |
Target Milestone|3.1.1 |3.2.0
------- Additional Comments From jm@jmason.org 2007-01-14 06:38 -------
reopening since those rules are not yet in 3.2.0, and we need to decide if they
will be or not
------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.
[Bug 4728] fix 20_dnsbl_tests.cf to use -lastexternal instead of -notfirsthop
Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4728
jm@jmason.org changed:
What |Removed |Added
----------------------------------------------------------------------------
Priority|P5 |P2
------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.
[Bug 4728] [review] DUL rules should only use the last external IP, not all but the first of the external IPs
Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4728
parkerm@pobox.com changed:
What |Removed |Added
----------------------------------------------------------------------------
Status Whiteboard|needs 1 vote |ready for commit
------- Additional Comments From parkerm@pobox.com 2006-02-02 16:37 -------
+1
------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.
[Bug 4728] DUL rules should use -firsttrusted not -notfirsthop
Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4728
------- Additional Comments From spamassassin@dostech.ca 2005-12-10 00:31 -------
No it's "-firsttrusted" or "-untrusted". I'm using it for now since it does
work in the case that you don't trust anyone else (which I really don't think
you should) and the mail is delivered to your MX and not your MSA (in which case
you get into that trusted but not internal crap -- SPF has similar issues bug 4661).
# If name is foo-firsttrusted, check only the Received header just
# after it enters our trusted networks; that's the only one we can
# trust the IP address from (since our relay added that header).
# And if name is foo-untrusted, check any untrusted IP address.
elsif ($set =~ /-(first|un)trusted$/)
{
my @tips = ();
foreach my $ip (@originating) {
if ($ip && !$trusted->contains_ip($ip)) {
push(@tips, $ip);
}
}
@ips = $self->ip_list_uniq_and_strip_private (@ips, @tips);
if ($1 eq "first") {
@ips = (defined $ips[0]) ? ($ips[0]) : ();
} else {
shift @ips;
}
}
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
[Bug 4728] DUL rules should use -firsttrusted not -notfirsthop
Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4728
------- Additional Comments From jm@jmason.org 2005-12-12 21:35 -------
> (I had to increase my rbl_timeout to 60 to get consistent sensible results)
60 seconds? eek!
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
[Bug 4728] fix 20_dnsbl_tests.cf to use -lastexternal instead of -notfirsthop
Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4728
jm@jmason.org changed:
What |Removed |Added
----------------------------------------------------------------------------
Status Whiteboard|ready for commit |pre-mass-check
------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.
[Bug 4728] [review] DUL rules should only use the last external IP, not all but the first of the external IPs
Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4728
jm@jmason.org changed:
What |Removed |Added
----------------------------------------------------------------------------
BugsThisDependsOn| |5294
------- Additional Comments From jm@jmason.org 2007-01-14 06:37 -------
T_E_RCVD_IN_XBL is now applied to replace RCVD_IN_XBL, as part of bug 5294.
The other T_E_* rules should similarly be applied ASAP, before we start the
3.2.0 mass-checks, or else left until 3.3.0.
------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.
[Bug 4728] [review] DUL rules should only use the last external IP, not all but the first of the external IPs
Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4728
spamassassin@dostech.ca changed:
What |Removed |Added
----------------------------------------------------------------------------
Status Whiteboard|needs 2 votes |needs 1 vote
------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.
[Bug 4728] DUL rules should use -firsttrusted not -notfirsthop
Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4728
------- Additional Comments From mkettler_sa@comcast.net 2005-12-10 00:21 -------
I assume "firsttrusted" is a typo, and you mean "firstuntrusted".
That said, "firstuntrusted" wouldn't work all that well either.
Really none of the features built into SA 3.1.0 would work very well for this.
"notfirsthop" (as currently done in 3.1.0) breaks for this (somewhat rare, but
legitimate) case:
public IP -> public (dyn) -> ISP -> Recipient MX -> SA.
While it is weird, they shouldn't be penalized with DUL hits since they did
relay through the ISP server. Admittedly this could be caused by dialup-joe
running an open-relay MTA on his home machine, but that's an issue for the
open-relay lists, not the DUL lists.
"firstuntrusted" breaks for the case that a user decides to trust an outside
ISPs MX to be non-forging. It seems perfectly plausible to trust such a relay to
not forge headers. Why should "trust" imply "part of my network that should not
get DUL mail"? The current docs actually outright tell SA users to declare their
MXes which accept direct dul mail as trusted but not internal.
Since this is really not so much a matter of trust vs untrust but internal vs
external, it would be better to do something along the lines of "firstexternal".
Consider this as the "worst case":
public IP (external,untrusted) ->
public (DUL, external, untrusted) ->
sender's ISP smarthost (external, trusted or untrusted) ->
Recipient's MX (internal, trusted) ->
SA (internal, trusted)
The only IP that should be checked against DULs is the sender's ISP smarthost,
since that's the box that dropped the mail off at our recipient's MX.
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
[Bug 4728] DUL rules should use -firsttrusted not -notfirsthop
Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4728
spamassassin@dostech.ca changed:
What |Removed |Added
----------------------------------------------------------------------------
Target Milestone|Undefined |3.1.1
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
[Bug 4728] [review] DUL rules should only use the last external IP, not all but the first of the external IPs
Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4728
spamassassin@dostech.ca changed:
What |Removed |Added
----------------------------------------------------------------------------
CC|spamassassin@dostech.ca |dev@spamassassin.apache.org
------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.