You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spamassassin.apache.org by bu...@bugzilla.spamassassin.org on 2006/03/16 23:03:59 UTC
[Bug 4828] New: DCC plugin cannot be configured to use ddcifd rcvd-nxt option
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4828
Summary: DCC plugin cannot be configured to use ddcifd rcvd-nxt
option
Product: Spamassassin
Version: SVN Trunk (Latest Devel Version)
Platform: Other
OS/Version: other
Status: NEW
Severity: normal
Priority: P5
Component: Plugins
AssignedTo: dev@spamassassin.apache.org
ReportedBy: sidney@sidney.com
dccifd takes an option 'rcvd-nxt' [see man dccifd(8)] that is needed when there
are Received headers for internal servers that need to be skipped.
dccifd takes n>=0 rcvd-nxt options to skip n Received headers.
We need a config option for the DCC plugin to specify n.
I'm entering this as a bug, not an RFE, because it is causing DCC to never match
at some sites.
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
[Bug 4828] DCC plugin cannot be configured to use ddcifd rcvd-nxt option
Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4828
sidney@sidney.com changed:
What |Removed |Added
----------------------------------------------------------------------------
Status Whiteboard|Needs 2 votes for 3.1 branch|
Target Milestone|3.1.2 |3.2.0
------- Additional Comments From sidney@sidney.com 2006-04-08 17:05 -------
Yes, I agree with you, Justin. As long as DCC's default is to use a Received
header that changes with each receiving ISP, they can't expect the information
to be useful and it won't matter what we do.
I see I accidentally closed this as fixed. I'll leave it that way and retarget
it to 3.2.
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
[Bug 4828] DCC plugin cannot be configured to use ddcifd rcvd-nxt option
Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4828
------- Additional Comments From spamassassin@dostech.ca 2006-03-17 00:44 -------
(In reply to comment #7)
> (In reply to comment #6)
> > Daryl, is it possible and valid to have unparseable relays before the Received
> > header for last internal relay?
>
> No, it is not. All trusted received headers (including internal ones) must be
> continuously parsable.
Just call me two comment Daryl...
I believe it is possible for some Qmail headers to be skipped though.
So I guess you'd have to pass it the other info in that case (I haven't read the
docs you refer to).
In either case... adding the generic config options like Kelsey suggests is a
good idea. After all, we already do it for Razor and/or Pyzor.
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
[Bug 4828] DCC plugin cannot be configured to use ddcifd rcvd-nxt option
Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4828
sidney@sidney.com changed:
What |Removed |Added
----------------------------------------------------------------------------
Attachment #3417 is|0 |1
obsolete| |
------- Additional Comments From sidney@sidney.com 2006-04-08 15:26 -------
Created an attachment (id=3463)
--> (http://issues.apache.org/SpamAssassin/attachment.cgi?id=3463&action=view)
patch to pass helo arguments to dccifd based on the Received header that
SpamAssassin has picked
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
[Bug 4828] DCC plugin cannot be configured to use ddcifd rcvd-nxt option
Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4828
sidney@sidney.com changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|RESOLVED |REOPENED
Resolution|FIXED |
------- Additional Comments From sidney@sidney.com 2006-04-10 14:27 -------
I asked about this on the DCC mailing list. I was wrong about what DCC is doing
and how we should use it. The summary of what I learned is:
The rcvd_next option is deprecated. DCC has configuration options MX and MXDCC
which are used to select the correct Received header in a similar manner to the
way SpamAssassin uses the trusted_networks configuration option. If DCC is
configured properly with MX and MXDCC it should use the same Received header as
SpamAssassin, but we can make that irrelevant by passing client ip address and
name and HELO string as in patch 3463. However, I missed the plugin's call to
dccproc which it uses if dccifd is not available. That call can be passed the
client ip address in a -a option. But dccproc needs the correct MX and MXDCC
options so it looks at the correct Received header for client hostname and HELO
string.
More importantly, I found that the DCC servers will block excessive use of
dccproc and therefore spamd should be rechecking for the availability of dccifd
regularly.
As the initial premise of this bug was wrong, I'm going to reopen this to change
it from FIXED to INVALID. See especially the first two links below for support
for my reasoning for that. The links are messages on the DCC mailing list. If we
need more changes to the plugin I'll open another bug.
http://www.rhyolite.com/pipermail/dcc/2006/003125.html
http://www.rhyolite.com/pipermail/dcc/2006/003126.html
http://www.rhyolite.com/pipermail/dcc/2006/003139.html
http://www.rhyolite.com/pipermail/dcc/2006/003140.html
http://www.rhyolite.com/pipermail/dcc/2006/003142.html
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
[Bug 4828] DCC plugin cannot be configured to use ddcifd rcvd-nxt option
Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4828
------- Additional Comments From kgc@sonic.net 2006-03-16 23:35 -------
Created an attachment (id=3417)
--> (http://issues.apache.org/SpamAssassin/attachment.cgi?id=3417&action=view)
patch to add support for passing extra options to dccifd configured in local.cf
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
[Bug 4828] DCC plugin cannot be configured to use ddcifd rcvd-nxt option
Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4828
------- Additional Comments From sidney@sidney.com 2006-03-17 00:13 -------
Daryl, is it possible and valid to have unparseable relays before the Received
header for last internal relay?
I just realized that I may have read the dccifd man page wrong. It says
'[...]are taken from the first Received
header if it has the standard "name (name [IP address])..."
format (or N+first if there are N rcvd-nxt options)'
That means that it will not work if it doesn't see the correct header. I guess
that's what you are saying: The mail at some site may go through different paths
under different circumstances, with different numbers of internal servers.
Perhaps it might be better to explicitly pass dcifd the client and HELO options?
Does the plugin have access to those values? That way there would be no need to
mess with the number of Received headers to skip.
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
[Bug 4828] DCC plugin cannot be configured to use ddcifd rcvd-nxt option
Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4828
------- Additional Comments From spamassassin@dostech.ca 2006-03-16 23:52 -------
Bah -- of course that should be
"num_relays_internal" for "n" and not "num_relays_external" +
"num_relays_unparseable" like I wrote above.
Same goes for 3.1... s/num_relays_untrusted/num_relays_trusted/
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
[Bug 4828] DCC plugin cannot be configured to use ddcifd rcvd-nxt option
Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4828
------- Additional Comments From spamassassin@dostech.ca 2006-04-11 02:30 -------
I'm a little confused as to why this was marked invalid AND the code from
r392533 (comment 13) was left as is.
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
[Bug 4828] DCC plugin cannot be configured to use ddcifd rcvd-nxt option
Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4828
------- Additional Comments From sidney@sidney.com 2006-03-17 03:45 -------
I just noticed that we already have a dcc_options for DCC, just nothing for
dccifd. dcc_options cleans the input to make sure it only contains legal
characters. For security reasons I would want the same in a dcc_dccifd_options
addition.
I'm still not sure about this. If it is the case that you cannot know for sure
what 'n' will be, what sense does it make to configure a fixed number of
rcvd_nxt options? And I don't see what other options could be used and still let
dccifd to work with SpamAssassin.
The choices are spam, body, header, query, grey-query, no-reject, and rcvd-nxt.
Of those, spam makes no sense (report all email as spam); header is always
included; body contradicts header and would not work; grey-query and no-reject
would prevent a return code that SA would recognize; query would not break
things, but could be accomplished by running the dcc daemon with the -Q option;
and rcvd-nxt is what we are talking about whether to use.
If we have access to the SMTP client ip address and host name and the EHLO
string, why would it not be better to always pass them along instead of forcing
dccifd to find the correct Received header and hope that it parses them correctly?
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
[Bug 4828] DCC plugin cannot be configured to use ddcifd rcvd-nxt option
Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4828
------- Additional Comments From spamassassin@dostech.ca 2006-03-16 23:49 -------
(In reply to comment #0)
> We need a config option for the DCC plugin to specify n.
Absolutely not. "n" isn't a static number.
We need to use "num_relays_external" + "num_relays_unparseable" for "n". A
config option isn't needed, or useful.
Note: "num_relays_external" isn't yet back-ported to 3.1, but
"num_relays_untrusted" will work for most cases (at least as well as it is for
other things now).
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
[Bug 4828] DCC plugin cannot be configured to use ddcifd rcvd-nxt option
Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4828
sidney@sidney.com changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |RESOLVED
Resolution| |FIXED
Status Whiteboard| |Needs 2 votes for 3.1 branch
Target Milestone|Undefined |3.1.2
------- Additional Comments From sidney@sidney.com 2006-04-08 15:46 -------
As far as I can tell there are no circumstances in which any of the options to
dccifd, other than the "header" option which we always send, would be correct
for SpamAssassin to use. A fixed number of rcvd-next would not work in the face
of variable length paths through internal relays taken by different emails. So
the only way of ensuring correct values passed to dccifd is to use values from
the Received header that SpamAssassin figures out, as in this patch.
Committed to trunk revision 392533.
I'm leaving this open retargeted to 3.1.2 because I'm concerned about the
possible effect of many SpamAssassin installations that are reporting to DCC
servers each with different helo and client values based on their first Received
headers, making that information useless for the clearinghouse checksum.
Comments and/or votes, please?
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
[Bug 4828] DCC plugin cannot be configured to use ddcifd rcvd-nxt option
Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4828
spamassassin@dostech.ca changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |spamassassin@dostech.ca
------- Additional Comments From spamassassin@dostech.ca 2006-03-17 06:16 -------
(In reply to comment #9)
> I'm still not sure about this. If it is the case that you cannot know for sure
> what 'n' will be, what sense does it make to configure a fixed number of
> rcvd_nxt options? And I don't see what other options could be used and still let
> dccifd to work with SpamAssassin.
Yeah, there's not much point in a static config option for rcvd_nxt. I'd be for
automatically passing a value for it if it weren't for us completely ignoring
some Qmail-like internal relays that provide no useful info.
> If we have access to the SMTP client ip address and host name and the EHLO
> string, why would it not be better to always pass them along instead of forcing
> dccifd to find the correct Received header and hope that it parses them correctly?
I've now read the current documentation
(http://www.rhyolite.com/anti-spam/dcc/dcc-tree/dccifd.html) and now have two
half clues about what you're talking about.
I think the best thing to do, given that we can't always be sure about the value
for "rcvd-nxt", is to pass "client" and "HELO" values using the values from the
first external received header. You can get them easily in the plugin via:
$permsgstatus->{relays_external}->[0]->{ip}
$permsgstatus->{relays_external}->[0]->{helo}
and an optional hostname for "client":
$permsgstatus->{relays_external}->[0]->{rdns}
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
[Bug 4828] DCC plugin cannot be configured to use ddcifd rcvd-nxt option
Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4828
------- Additional Comments From spamassassin@dostech.ca 2006-03-17 00:35 -------
(In reply to comment #6)
> Daryl, is it possible and valid to have unparseable relays before the Received
> header for last internal relay?
No, it is not. All trusted received headers (including internal ones) must be
continuously parsable.
> I just realized that I may have read the dccifd man page wrong. It says
>
> '[...]are taken from the first Received
> header if it has the standard "name (name [IP address])..."
> format (or N+first if there are N rcvd-nxt options)'
>
> That means that it will not work if it doesn't see the correct header. I guess
> that's what you are saying: The mail at some site may go through different paths
> under different circumstances, with different numbers of internal servers.
Correct, for instance, my own mail can be delivered directly to the machine
running SA, or pass through about 8 additional relays that are considered
internal. Even for the simple case of a secondary MX that passed mail to a
primary MX which scans the mail, "n" will vary between 0 and 1.
> Perhaps it might be better to explicitly pass dcifd the client and HELO options?
> Does the plugin have access to those values? That way there would be no need to
> mess with the number of Received headers to skip.
The plugin has access to all of the data parsed from the received headers, but
just passing it "n" for the number of internal relays will work most reliably
and is conveniently the easiest to implement.
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
[Bug 4828] DCC plugin cannot be configured to use ddcifd rcvd-nxt option
Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4828
------- Additional Comments From jm@jmason.org 2006-04-08 16:06 -------
I'd prefer to avoid changing 3.1.2 unless someone from DCC suggests it'd be a
good idea ;)
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
[Bug 4828] DCC plugin cannot be configured to use ddcifd rcvd-nxt option
Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4828
------- Additional Comments From sidney@sidney.com 2006-04-08 12:54 -------
I tested with a patch that uses
$permsgstatus->{relays_external}->[0]->{ip}
$permsgstatus->{relays_external}->[0]->{helo}
$permsgstatus->{relays_external}->[0]->{rdns}
Looking at the dccifd logs, as far as I can tell this makes no difference to
three checksums that are actually used for DCC checking, Body, Fuz1 and Fuz2.
That makes sense, because without that patch the wrong Received header and
therefore the wrong values for dccifd "client" and "helo" protocol lines are
_always_ chosen, so every Spamassassin client has been using DCC with the wrong
values for those.
I'm inclined to check in a patch to trunk simply because it is more correct, but
I fail to see how this would have caused failures in DCC_CHECK anywhere.
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
[Bug 4828] DCC plugin cannot be configured to use ddcifd rcvd-nxt option
Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4828
------- Additional Comments From sidney@sidney.com 2006-04-11 02:52 -------
The bug report as I filed it was invalid, in that Kelsey was seeing DCC breakage
that was due to a DCC configuration error, not the need for the rcvd_next option.
However, the patch I checked in instead of using rcvd_next is something that I
think is needed for part of bug 4861. Even though it is only a partial solution,
it makes some things better without breaking anything new. So I didn't pull it
out when I closed this bug, instead I opened a new bug.
To document what is going on, I'll add a comment to bug 4861 pointing to this
comment and the patch.
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
[Bug 4828] DCC plugin cannot be configured to use ddcifd rcvd-nxt option
Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4828
kgc@sonic.net changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |kgc@sonic.net
------- Additional Comments From kgc@sonic.net 2006-03-16 23:32 -------
I've got a slightly more generalized patch that allows the administrator to
define whatever extra options should be passed to dccifd that the admin sees fit
as there are circumstances where access to change more than the presenece of one
or more rcvd-nxt options might be required.
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
[Bug 4828] DCC plugin cannot be configured to use ddcifd rcvd-nxt option
Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4828
------- Additional Comments From kgc@sonic.net 2006-03-16 23:59 -------
(In reply to comment #2)
> Created an attachment (id=3417)
--> (http://issues.apache.org/SpamAssassin/attachment.cgi?id=3417&action=view) [edit]
> patch to add support for passing extra options to dccifd configured in local.cf
Which is in production and functional.
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
[Bug 4828] DCC plugin cannot be configured to use ddcifd rcvd-nxt option
Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4828
sidney@sidney.com changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|REOPENED |RESOLVED
Resolution| |INVALID
------- Additional Comments From sidney@sidney.com 2006-04-10 14:28 -------
See comment #16
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.