You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spamassassin.apache.org by bu...@issues.apache.org on 2010/08/18 10:17:47 UTC

[Bug 6484] New: Mail::SpamAssassin::Plugin::ASN - Alternative zone FORMAT enhancements.

https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6484

           Summary: Mail::SpamAssassin::Plugin::ASN - Alternative zone
                    FORMAT enhancements.
           Product: Spamassassin
           Version: 3.3.1
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: enhancement
          Priority: P5
         Component: Plugins
        AssignedTo: dev@spamassassin.apache.org
        ReportedBy: kd6lvw+software@kd6lvw.ampr.org


Created an attachment (id=4800)
 --> (https://issues.apache.org/SpamAssassin/attachment.cgi?id=4800)
UNTESTED replacement for file ASN.pm (not a diff-style patch)

1) It appears that the TXT-RR returned by at least one alternative zone
offering has only one field, the ASN.  Since it lacks 3 fields, extracting the
data in process_dns_result() of the module FAILS.

2) I have also noted at least two other zones offering the same or similar
data, some with additional fields, and formatted differently.  I've rewritten
the process_dns_result() routine in the ASN.pm file to accomodate these
additional formats.

Zones being tested:
   asn.routeviews.org
   origin.asn.cymru.com
   origin.asn.spameatingmonkey.net
   ip2asn.sasm4.net

3)  I note that this plugin handles IPv4 lookups only.  I shall leave IPv6
handling for someone more skilled in SA and perl.  Meanwhile, I do note that
there is one IPv6 lookup zone that follows the "ip6.arpa" format for reversing
an IPv6 address:

   origin6.asn.cymru.com

The IPv6 modification which is needed appears to be ONLY in parsed_metadata()
after the "local()" where I have commented.

I am alpha-testing a replacement file now.  It passes the "SA --lint" test. 
However, I don't think it's working.  However, it did fetch these DNS TXT-RR's
for my test message:

123.237.142.68.ip2asn.sasm4.net. 3501 TXT "as14779"
123.237.142.68.asn.routeviews.org. 86300 TXT "14779" "68.142.224.0" "20"
123.237.142.68.origin.asn.cymru.com. 14300 TXT "14779 | 68.142.224.0/20 | US |
arin | 2004-03-24"
123.237.142.68.origin.asn.spameatingmonkey.net. 200 TXT "68.142.224.0/20 |
AS14779 | Inktomi Corporation | 2000-02-07 | Unknown"

(Test source address 68.142.237.123 is a Yahoo.com mail server.  All four
records were in my DNS server's cache, so I know the lookup part still works.)

Does anyone else want to take a stab at upgrading this module (and share
credit)?

-- 
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 6484] Mail::SpamAssassin::Plugin::ASN - Alternative zone FORMAT enhancements.

Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6484

--- Comment #15 from Mark Martinec <Ma...@ijs.si> 2010-12-15 15:09:32 UTC ---
> Patch revised again - now calling the common reverse_ip_address() subroutine
> just installed into Util.pm.
> 
> Patch passes "lint" operation.  Time to [beta] check for correctness and
> commit.

I have some changes for that plugin on my own too, just need to merge the two.

-- 
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 6484] Mail::SpamAssassin::Plugin::ASN - Alternative zone FORMAT enhancements.

Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6484

--- Comment #12 from Mark Martinec <Ma...@ijs.si> 2010-12-14 21:24:46 UTC ---
> Since this repeats code, we should probably have a
> subroutine/function call for computing the reverse

Done.

trunk:
  Bug 6484: factor out common code as SpamAssassin::Util::reverse_ip_address
Sending lib/Mail/SpamAssassin/Plugin/DNSEval.pm
Sending lib/Mail/SpamAssassin/Util.pm
Committed revision 1049391.

-- 
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 6484] Mail::SpamAssassin::Plugin::ASN - Alternative zone FORMAT enhancements.

Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6484

Karsten Bräckelmann <gu...@rudersport.de> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |REOPENED
         Resolution|WORKSFORME                  |

--- Comment #3 from Karsten Bräckelmann <gu...@rudersport.de> 2010-09-09 14:05:37 UTC ---
(In reply to comment #2)
> I believe I am done with this suggestion.  Please beta test the patch, and if
> no problems are found, consider it for inclusion with SA 3.3.2.  Thanks

A bug report is not resolved, because a patch you're comfortable with is
attached. If the code is desired to be included upstream, it has to be
committed to trunk at the very least.

Moreover, Resolution WORKSFORME means, that the reported bug cannot be
reproduced and appears to not be a bug. Please see the "Status" link in the
details above. :)

Reopening.

-- 
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 6484] [review] Mail::SpamAssassin::Plugin::ASN - Alternative zone FORMAT enhancements.

Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6484

--- Comment #32 from Darxus <Da...@ChaosReigns.com> 2011-05-27 21:32:00 UTC ---
(In reply to comment #30)
> OK, I'll yield on the 3.3.2 issue - with one last comment.  The DNS database at

That's everybody.  Please close.  Already committed to target, and nobody is
going to change the target.

-- 
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 6484] [review] Mail::SpamAssassin::Plugin::ASN - Alternative zone FORMAT enhancements.

Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6484

--- Comment #31 from Kevin A. McGrail <km...@pccc.com> 2011-05-27 21:23:55 UTC ---
(In reply to comment #30)
> OK, I'll yield on the 3.3.2 issue - with one last comment.  The DNS database at
> routeviews IS one of the buggy ones.  However, without the patch, IPv6
> addresses won't be checked, so no harm there.
> 
> I am using this patch with my 3.3.2-rc1.

I see this as a pretty substantial change better for a 3.4 release.  I also am
concerned it is non trivial and you should get a CLA filed so we can freely
apply your patch.  I might be overly concerned here.

http://www.apache.org/licenses/icla.pdf

Regards,
KAM

-- 
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 6484] [review] Mail::SpamAssassin::Plugin::ASN - Alternative zone FORMAT enhancements.

Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6484

--- Comment #27 from D. Stussy <so...@kd6lvw.ampr.org> 2011-05-27 19:12:17 UTC ---
Considering the discussion on the IETF message list, I suggest the following
course of action:

1)  Commit that which was committed to trunk also to 3.3.2 - as if it were IPv4
only (even though its not).  That is, ignore the problem for now.  We will have
to accept the problem and perhaps NOT recommend usage of the DNS zones which
have the problem.  Since this patch also allows MULTIPLE zone lookups, that's
the functionality I'm suggesting for 3.3.2.  The patch has changes to both
Util.pm and Plugin/ASN.pm.

2)  Redesignate any further IPv6 fix for version 3.4 - awaiting an RFC which
fixes the issue.  (New bug # if agreed upon.  Create a "todo" list file?)

3)  In the meantime, we could try some sort of "quick and dirty" additional
patch:  If the queried address has a colon, only accept data which has a colon
in the network address range.


Also noted:  The relay-country plugin in the example ASN failure came up with
"XX" (for the first relay) while one of the DNS queries returned additional
fields, one of which had the missing country.  If we can guarentee that the two
plugins are examining the same "first" ip address, we have a chance to take
corrective action (which I suggest ONLY when the first relay-country is "XX"). 
Of course, we also have to validate that the third ASN response field is two
upper-case letters only (as it is for cymru.com's results) before replacing
"^XX" [note the regex] with something else.

-- 
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 6484] Mail::SpamAssassin::Plugin::ASN - Alternative zone FORMAT enhancements.

Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6484

Kevin A. McGrail <km...@pccc.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |kmcgrail@pccc.com

--- Comment #10 from Kevin A. McGrail <km...@pccc.com> 2010-12-14 16:33:03 UTC ---
(In reply to comment #9)
> Created an attachment (id=4834)
 --> (https://issues.apache.org/SpamAssassin/attachment.cgi?id=4834) [details]
> Revised Patch - IPv6 checks for NetAddr::IP >= 4.000
> 
> Added check for proper version of NetAddr::IP (>= 4.000), also borrowed from
> bug 6519.  However, what we really should have is a subroutine to call that
> converts for us as this code is repeated in at least two if not more places.
> 
> Short of a common routine, this is commit-ready.

Having just dealt with an issue where two places implemented the same routine
in SA and the code forked, I'd really like to see this code unified before more
commits occur.

-- 
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 6484] [review] Mail::SpamAssassin::Plugin::ASN - Alternative zone FORMAT enhancements.

Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6484

--- Comment #26 from Darxus <Da...@ChaosReigns.com> 2011-05-27 17:07:07 UTC ---
Discussion of IPv4 and IPv6 DNSBL overlap on the ASRG list: 
http://www.ietf.org/mail-archive/web/asrg/current/msg16843.html

I'm pretty disappointed so far.

-- 
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 6484] Mail::SpamAssassin::Plugin::ASN - Alternative zone FORMAT enhancements.

Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6484

software+spamassassin@kd6lvw.ampr.org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |software+spamassassin@kd6lv
                   |                            |w.ampr.org

--- Comment #8 from software+spamassassin@kd6lvw.ampr.org 2010-12-13 14:07:17 UTC ---
"0:0:0:0:0:FFFF:192.0.2.1" is definently not canonical and per some RFCs not
even valid, because where there are multiple 0-quads, the longest group [2 or
more consecutive] MUST be collapsed to "::" - i.e. compressed form is required
where possible (RFC 5952, Section 4.2.1 - which updates RFC 4291).  As the IPv4
embedded forms which express the IPv4 address in its canonical form
("alternative form") lie both within ::/80, there will be at most 6 separators.
 Although IPv4 addresses lie within 2002::/16, they are not expressed as such
but remain IPv6 quads.

Per RFC 5952, your "alternative forms" are not valid, and therefore, I see no
need to expand the regex to validate them, even if they are translatable into
equivalent valid forms.

-- 
Note:  I am in agreement with all of RFC 5952 except section 4.3 as that
section contradicts 50+ years of computer science history.  Canonical form for
any non-decimal hexadecimal digit (or any radix 11-36) is an upper-case
alphabetic, regardless of context.

-- 
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 6484] [review] Mail::SpamAssassin::Plugin::ASN - Alternative zone FORMAT enhancements.

Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6484

Darxus <Da...@ChaosReigns.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |Darxus@ChaosReigns.com

--- Comment #24 from Darxus <Da...@ChaosReigns.com> 2011-05-26 18:55:39 UTC ---
D. Stussy, can you give me an example of a DNSRBL that is returning values for
IPv4 2.0.0.0/8 for IPv6 queries in 2000/16?

I understand the problem.  I'm wondering about digging up some evidence and
talking to the folks who did the RFC: 
http://tools.ietf.org/html/rfc5782#section-2.4

I think a better solution is to add something to the IPv6 records.  So instead
of [reversed IP].[zone], it's something like [reversed IP].v6.[zone].


Looks like this should be closed:  It's committed to trunk, which it's targeted
for.  Unless somebody wants to re-target it for 3.3.x.

-- 
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 6484] Mail::SpamAssassin::Plugin::ASN - Alternative zone FORMAT enhancements.

Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6484

software+spamassassin@kd6lvw.ampr.org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Attachment #4834|0                           |1
        is obsolete|                            |

--- Comment #14 from software+spamassassin@kd6lvw.ampr.org 2010-12-15 14:39:49 UTC ---
Created an attachment (id=4836)
 --> (https://issues.apache.org/SpamAssassin/attachment.cgi?id=4836)
Revised Patch - Calls common sub reverse_ip_address() in Util.pm

Patch revised again - now calling the common reverse_ip_address() subroutine
just installed into Util.pm.

Patch passes "lint" operation.  Time to [beta] check for correctness and
commit.

-- 
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 6484] [review] Mail::SpamAssassin::Plugin::ASN - Alternative zone FORMAT enhancements.

Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6484

--- Comment #25 from D. Stussy <so...@kd6lvw.ampr.org> 2011-05-26 20:43:45 UTC ---
I had to review my saved spam to find one. From May 3, 2011:

ASN: 3215 29222 (2.0.0.0/16 2001:1600::/32) XX ES
CIP: 2001:1600:2:5:21d:9ff:fe65:e562 smtp2.infomaniak.ch

Where those lines in my report are:

report ASN: _ASN_ (_ASNCIDR_) _RELAYCOUNTRY_
report CIP: _LASTEXTERNALIP_ _LASTEXTERNALRDNS_

asn_lookup origin6.asn.cymru.com - is the line returning the CORRECT info:
2.6.5.e.5.6.e.f.f.f.9.0.d.1.2.0.5.0.0.0.2.0.0.0.0.0.6.1.1.0.0.2.origin6.asn.cymru.com.
14400 IN TXT "29222 | 2001:1600::/32 | CH | ripencc | 2007-07-26"

FAIL asn_lookup origin.asn.cymru.com
FAIL asn_lookup asn.routeviews.org
PASS asn_lookup ip2asn.sasm4.net                 (noerror to ANY query)
PASS asn_lookup origin.asn.spameatingmonkey.net  (nxdomain)

Incorrect answers:
2.6.5.e.5.6.e.f.f.f.9.0.d.1.2.0.5.0.0.0.2.0.0.0.0.0.6.1.1.0.0.2.asn.routeviews.org.
43200 IN TXT "3215" "2.0.0.0" "16"
2.6.5.e.5.6.e.f.f.f.9.0.d.1.2.0.5.0.0.0.2.0.0.0.0.0.6.1.1.0.0.2.origin.asn.cymru.com.
14400 IN TXT "3215 | 2.0.0.0/16 | FR | ripencc | 2010-07-12"


The CORRECT response is:
ASN: 29222 (2001:1600::/32) XX ES

This info is wrong but included due to the bogus DNSBL responses:
ASN: 3215 (2.0.0.0/16)

PS:  I would also like this to appear in 3.3.2.  Should we fix the problem
first?

-- 
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 6484] Mail::SpamAssassin::Plugin::ASN - Alternative zone FORMAT enhancements.

Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6484

Kevin A. McGrail <km...@pccc.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|3.4.0                       |3.4.1

--- Comment #37 from Kevin A. McGrail <km...@pccc.com> ---
Moving all open bugs where target is defined and 3.4.0 or lower to 3.4.1 target

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 6484] Mail::SpamAssassin::Plugin::ASN - Alternative zone FORMAT enhancements.

Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6484

Mark Martinec <Ma...@ijs.si> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Status Whiteboard|needs 2 votes (for 3.4)     |needs fixing the remaining
                   |                            |issue

-- 
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 6484] [review] Mail::SpamAssassin::Plugin::ASN - Alternative zone FORMAT enhancements.

Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6484

--- Comment #33 from D. Stussy <so...@kd6lvw.ampr.org> 2011-05-27 21:44:34 UTC ---
Not so fast:  We still have an issue here.  It's just that we all agree not to
fix it for 3.3.2 - targeted for 3.4.0.  See comment #23 for a solution idea.

-- 
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 6484] Mail::SpamAssassin::Plugin::ASN - Alternative zone FORMAT enhancements.

Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6484

--- Comment #36 from D. Stussy <so...@kd6lvw.ampr.org> 2011-06-27 00:02:29 UTC ---
I have a solution to the problem of the IPv4/IPv6 lookup overlap (which will
hold true for other DNS based lists too).  It involves the DNS DNAME-RR. 
However, for the FULL IPv6 address space, it may not scale well as it would
require 16,666 DNAME entries in a given zone.  There is NO need to further
modify SA for this solution, but SA should still be modified so as not to make
unnecessary queries (i.e. to IPv4-only or IPv6-only databases).


The Solution - DNS records only.  Takes avantage of two factors:
1)  Some IPv6 labels in the reverse zone are alphabetic -- which never appears
in an IPv4 reverse.
2)  The nature of the DNAME-RR once we get to 4 specified label levels.

Let's call the IPv4 based list "dnsbl4.example.org."  Let's let the IPv6 based
list "dnsbl6.example.org" also exist.

First, let's eliminate/redirect the last six labels of the IPv6 address range.

(Using BIND syntax):
$ORIGIN dnsbl4.example.org.
;
; 6 labels take care of the top 37.5% of IPv6 space:
;
A        IN  DNAME   A.dnsbl6.example.org.
...
F        IN  DNAME    F.dnsbl6.example.org.
;
; Repeating this for the second nybble does scale for BIND.
; Other DNS software - you're on your own....
; This adds 60 more DNAME-RR's (23.4375% coverage of IPv6 space)
;
$GENERATE 0-9 A.$  IN  DNAME  A.$.dnsbl6.example.org.
...
$GENERATE 0-9 F.$  IN  DNAME  F.$.dnsbl6.example.org.
;

Repeating that with a two-level generate (if permitted; else by hand) for 600
more records takes care of all 3-label levels.  (Another 14.648%)

At the 4th label level, we have 16,000 DNAMES generated (16 x 10^3), and in
that leading label (the least significant), we generate all 16 hexidecimal
digits (the remaining 24.414%).  Such yields entries such as:

1.0.0.2    IN  DNAME   1.0.0.2.dnsbl6.example.org.
2.0.0.2    IN  DNAME   2.0.0.2.dnsbl6.example.org.
3.0.0.2    IN  DNAME   3.0.0.2.dnsbl6.example.org.
...

Since DNAME implicitly requires a label below that specified (as it applies to
subzones only), this does not conflict with IPv4 addresses which would
terminate here.  However, it does mean that IPv4 addresses NOT listed which
overlap with IPv6 prefixes may no longer return "NXDOMAIN" but may return
"NOERROR" with zero answers.  Hopefully, no DNS-based list query (including
DNSBLs and DNSWLs) would react differently.

I have also posted this idea to Usenet and maybe we should have comments there?
cf. Newsgroups:
news.admin.net-abuse.email,news.admin.net-abuse.misc,alt.ipv6,alt.internet.ipv6,alt.comp.networking.ipv6,alt.dns

Since we currently don't have to cover 100% of IPv6 address space at this time,
we could get by with fewer than the 16,666 DNAME RR's needed for full coverage.

-- 
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 6484] [review] Mail::SpamAssassin::Plugin::ASN - Alternative zone FORMAT enhancements.

Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6484

--- Comment #23 from software+spamassassin@kd6lvw.ampr.org 2011-05-17 02:51:35 UTC ---
I note committed to trunk, but I don't seem to see this in 3.3.2-rc1; correct?

I have now also noted one problem:  Some IPv4-only DNS databases when queried
with an IPv6 address are returning data appropriate for the IPv4 subnet
2.0.0.0/8 (since all IPV6's looked up are in 2000::/3).  Maybe we need to an
additional indicator to mark such databases as IPv4 or IPv6 capable.  Two ways
of doing this:

1)  A separate directive for IPv6 databases:  asn6_lookup

2)  An additional parameter for defining databases:  asn_lookup [46] _database_
...,  where the "[46]" contains one or both numbers as flags to indicate the
appropriate address family.  If omitted, assume IPv4 only.

Either way, internally, we need to store a flag field per database and query
only those databases with the matching address family.

I think that to do this efficiently is beyond my perl abilities at this time.

-- 
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 6484] [review] Mail::SpamAssassin::Plugin::ASN - Alternative zone FORMAT enhancements.

Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6484

--- Comment #30 from D. Stussy <so...@kd6lvw.ampr.org> 2011-05-27 21:19:45 UTC ---
OK, I'll yield on the 3.3.2 issue - with one last comment.  The DNS database at
routeviews IS one of the buggy ones.  However, without the patch, IPv6
addresses won't be checked, so no harm there.

I am using this patch with my 3.3.2-rc1.

-- 
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 6484] [review] Mail::SpamAssassin::Plugin::ASN - Alternative zone FORMAT enhancements.

Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6484

--- Comment #22 from software+spamassassin@kd6lvw.ampr.org 2011-03-11 18:03:31 EST ---
2 months of beta-testing on two different systems and no problems noted....
Trunk change 1049391 affirmed as valid here.

-- 
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 6484] Mail::SpamAssassin::Plugin::ASN - Alternative zone FORMAT enhancements.

Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6484

--- Comment #13 from Mark Martinec <Ma...@ijs.si> 2010-12-14 21:27:32 UTC ---
Created an attachment (id=4835)
 --> (https://issues.apache.org/SpamAssassin/attachment.cgi?id=4835)
trunk change 1049391 - attached as a patch

-- 
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 6484] [review] Mail::SpamAssassin::Plugin::ASN - Alternative zone FORMAT enhancements.

Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6484

--- Comment #29 from Kevin A. McGrail <km...@pccc.com> 2011-05-27 20:24:59 UTC ---
(In reply to comment #28)
> I think it would be better to wrap up 3.3.2 as is, and start on 3.4.0, than to
> add stuff like this to 3.3.
> 
> And FYI, the guy who responded on the ASRG list (and who I think runs it) is
> listed as the author of the RFC with the IPv4/v6 DNSBL overlap problem.

I concur.  This patch is not ok for 3.3.2 IMO.  3.3.2 for me is a release to
fix the perl 5.12.X warnings primarily.  It's also been a great test case to
get all the ducks in a row for a release so hopefully releases like 3.4.0 won't
be too far distant.

-- 
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 6484] Mail::SpamAssassin::Plugin::ASN - Alternative zone FORMAT enhancements.

Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6484

--- Comment #35 from D. Stussy <so...@kd6lvw.ampr.org> 2011-06-03 00:18:19 UTC ---
Note:  Part of the code of the patch has already been committed to trunk, and
part not.  I think we should commit the remainder to trunk so everyone working
on the problem have an agreed-upon common starting point.

What we have (or is already pending):  The code to read multiple DNS database
results in varying formats.

What we need:  A way to limit our queries for these databases to just the
address families supported by them (a configuration item).  There have been two
suggestions on how to approach that (summarized at comment #23).  Which way do
we all want to go?

-- 
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 6484] Mail::SpamAssassin::Plugin::ASN - Alternative zone FORMAT enhancements.

Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6484

--- Comment #7 from Mark Martinec <Ma...@ijs.si> 2010-12-13 10:45:45 UTC ---
> I noted in the regex for IPv6, we don't need 9 copies (but only 7) of the
> latter term as permitting 9 would allow ANY representation where the lower 32
> bits are represented as an IPv4 embedded address.  However, only two forms
> (in "::/80") were ever defined - and at most, they would have 3 colons and
> 3 dots (implying a maximum value of 6, but regular IPv6 addresses can have
> 7 colons). Therefore, 9 was lowered to 7 so as to exclude invalid
> representations.

Don't know where your understanding of the 'alternative form' comes from.

My understanding of RFC 4291 section 2.2 yields up to 10 fields,
e.g.:   0:0:0:0:0:FFFF:192.0.2.1


  An alternative form that is sometimes more convenient when dealing
  with a mixed environment of IPv4 and IPv6 nodes is
  x:x:x:x:x:x:d.d.d.d, where the 'x's are the hexadecimal values of
  the six high-order 16-bit pieces of the address, and the 'd's are
  the decimal values of the four low-order 8-bit pieces of the
  address (standard IPv4 representation).  Examples:
    0:0:0:0:0:0:13.1.68.3
    0:0:0:0:0:FFFF:129.144.52.38
  or in compressed form:
    ::13.1.68.3
    ::FFFF:129.144.52.38


It is not the purpose of this regexp to validate IPv6 addresses,
but just to prescreen/triage obvious misfits. The NetAddr::IP::full6
is perfectly capable of weeding out remaining invalid addresses.

-- 
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 6484] [review] Mail::SpamAssassin::Plugin::ASN - Alternative zone FORMAT enhancements.

Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6484

--- Comment #34 from Darxus <Da...@ChaosReigns.com> 2011-05-27 21:51:27 UTC ---
Okay, then please take "[review]" out of the summary, since that means a bug
just needs votes and a commit.

The remaining issue is IPv4/v6 overlap, a known problem with
http://tools.ietf.org/html/rfc5782#section-2.4

-- 
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 6484] Mail::SpamAssassin::Plugin::ASN - Alternative zone FORMAT enhancements.

Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6484

--- Comment #20 from software+spamassassin@kd6lvw.ampr.org 2011-01-06 17:23:39 UTC ---
Disregard my last regarding line 116.  Realized that my UTIL.pm wasn't
exporting reverse_ip_address().

-- 
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 6484] Mail::SpamAssassin::Plugin::ASN - Alternative zone FORMAT enhancements.

Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6484

--- Comment #18 from software+spamassassin@kd6lvw.ampr.org 2011-01-06 14:26:26 UTC ---
OK.  I agree with your additions (code untested, but quickly reviewed).  You
could have added yourself to the credits for the modification in the file
comments, but that's your choice.  I hadn't thought of the two additional
configuration options.  Good job.

-- 
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 6484] Mail::SpamAssassin::Plugin::ASN - Alternative zone FORMAT enhancements.

Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6484

Kevin A. McGrail <km...@pccc.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|[review]                    |Mail::SpamAssassin::Plugin:
                   |Mail::SpamAssassin::Plugin: |:ASN - Alternative zone
                   |:ASN - Alternative zone     |FORMAT enhancements.
                   |FORMAT enhancements.        |

-- 
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 6484] Mail::SpamAssassin::Plugin::ASN - Alternative zone FORMAT enhancements.

Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6484

Karsten Bräckelmann <gu...@rudersport.de> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|REOPENED                    |NEW

--- Comment #4 from Karsten Bräckelmann <gu...@rudersport.de> 2010-09-09 14:07:04 UTC ---
Reopened, back to NEW.

-- 
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 6484] Mail::SpamAssassin::Plugin::ASN - Alternative zone FORMAT enhancements.

Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6484

--- Comment #19 from software+spamassassin@kd6lvw.ampr.org 2011-01-06 16:41:56 UTC ---
Code lints.  Now testing it by running it.
I did have to change one line to make it work with my 3.3.1:  Line 116:

- use Mail::SpamAssassin::Util qw(reverse_ip_address);
+ use Mail::SpamAssassin::Util;

Dropped the "qw(reverse_ip_address)" clause.  Let's make certain that it
doesn't need to be dropped in the trunk version also.

-- 
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 6484] Mail::SpamAssassin::Plugin::ASN - Alternative zone FORMAT enhancements.

Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6484

--- Comment #6 from software+spamassassin@kd6lvw.ampr.org 2010-12-09 15:26:24 UTC ---
Created an attachment (id=4832)
 --> (https://issues.apache.org/SpamAssassin/attachment.cgi?id=4832)
Revised Patch - Handles IPv6 labels/addresses

I borrowed the code idea in 6519 (but not an exact copy) to extend the lookups
into IPv6 zones.  Since this repeats code, we should probably have a
subroutine/function call for computing the reverse, but as we don't have that
yet, I'd let this code stand.

I noted in the regex for IPv6, we don't need 9 copies (but only 7) of the
latter term as permitting 9 would allow ANY representation where the lower 32
bits are represented as an IPv4 embedded address.  However, only two forms (in
"::/80") were ever defined - and at most, they would have 3 colons and 3 dots
(implying a maximum value of 6, but regular IPv6 addresses can have 7 colons). 
Therefore, 9 was lowered to 7 so as to exclude invalid representations.

-- 
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 6484] Mail::SpamAssassin::Plugin::ASN - Alternative zone FORMAT enhancements.

Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6484

--- Comment #5 from kd6lvw+software@kd6lvw.ampr.org 2010-09-16 05:07:56 UTC ---
OK, Whatever.  I'm done with what I wrote and it may be committed.

However, as noted, this still doesn't handle IPv6 addresses.

-- 
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 6484] Mail::SpamAssassin::Plugin::ASN - Alternative zone FORMAT enhancements.

Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6484

software+spamassassin@kd6lvw.ampr.org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|kd6lvw+software@kd6lvw.ampr |
                   |.org                        |

--- Comment #11 from software+spamassassin@kd6lvw.ampr.org 2010-12-14 19:33:24 UTC ---
I agree.  However, that shall require that one of the project maintainers
extract the duplicated code, write the common routine, and decide where it
should be placed.  That's not my job.  All I can do (and have done) is to
identify the duplicated section and indicate that such compaction is needed.

-- 
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 6484] Mail::SpamAssassin::Plugin::ASN - Alternative zone FORMAT enhancements.

Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6484

--- Comment #21 from software+spamassassin@kd6lvw.ampr.org 2011-01-06 17:24:24 UTC ---
Disregard my last regarding line 116.  Realized that my UTIL.pm wasn't
exporting reverse_ip_address().  It appears to be fully working so far now that
I've seen about 5 messages go through.

-- 
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 6484] Mail::SpamAssassin::Plugin::ASN - Alternative zone FORMAT enhancements.

Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6484

kd6lvw+software@kd6lvw.ampr.org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
                 CC|                            |kd6lvw+software@kd6lvw.ampr
                   |                            |.org
         Resolution|                            |WORKSFORME

--- Comment #2 from kd6lvw+software@kd6lvw.ampr.org 2010-09-08 19:31:34 UTC ---
I believe I am done with this suggestion.  Please beta test the patch, and if
no problems are found, consider it for inclusion with SA 3.3.2.  Thanks

-- 
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 6484] Mail::SpamAssassin::Plugin::ASN - Alternative zone FORMAT enhancements.

Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6484

--- Comment #17 from Mark Martinec <Ma...@ijs.si> 2011-01-06 07:17:35 UTC ---
trunk:

Plugin/ASN.pm overhaul:
- Bug 6484: alternative zone FORMAT enhancements
- recognize response formats using multiple TXT RR string fields
- use new SA::Util::reverse_ip_address() utility
- allow ASN queries for IPv6 addresses
- added config directives clear_asn_lookups and asn_prefix
- use a public API routine $pms->set_tag() for setting
  tag values instead of directly accessing data structure
- cleanup, some hardening

Sending lib/Mail/SpamAssassin/Plugin/ASN.pm
Committed revision 1055846.

-- 
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 6484] [review] Mail::SpamAssassin::Plugin::ASN - Alternative zone FORMAT enhancements.

Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6484

Mark Martinec <Ma...@ijs.si> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Priority|P5                          |P3
   Target Milestone|3.3.2                       |3.4.0
            Summary|Mail::SpamAssassin::Plugin: |[review]
                   |:ASN - Alternative zone     |Mail::SpamAssassin::Plugin:
                   |FORMAT enhancements.        |:ASN - Alternative zone
                   |                            |FORMAT enhancements.
  Status Whiteboard|                            |needs 2 votes (for 3.4)

-- 
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 6484] [review] Mail::SpamAssassin::Plugin::ASN - Alternative zone FORMAT enhancements.

Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6484

--- Comment #28 from Darxus <Da...@ChaosReigns.com> 2011-05-27 19:46:07 UTC ---
I think it would be better to wrap up 3.3.2 as is, and start on 3.4.0, than to
add stuff like this to 3.3.

And FYI, the guy who responded on the ASRG list (and who I think runs it) is
listed as the author of the RFC with the IPv4/v6 DNSBL overlap problem.

-- 
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 6484] Mail::SpamAssassin::Plugin::ASN - Alternative zone FORMAT enhancements.

Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6484

--- Comment #16 from software+spamassassin@kd6lvw.ampr.org 2010-12-15 16:23:55 UTC ---
OK.  What do you have in mind?  If it's not a trivial change, maybe it's best
to commit this and open a new bug/enhancement, especially if it deals with a
different section?

I hope this isn't one of the reasons 3.3.2 hasn't been pushed out (to correct
all those perl 5.12 warnings).  ;-)

Same with #6490 (adds detection for SPF=none, and RFC 5451 header parsing).  I
believe it's compatible with #5684 if both are applied (although I voiced
opposition to that patch).

-- 
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 6484] Mail::SpamAssassin::Plugin::ASN - Alternative zone FORMAT enhancements.

Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6484

software+spamassassin@kd6lvw.ampr.org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Attachment #4832|0                           |1
        is obsolete|                            |

--- Comment #9 from software+spamassassin@kd6lvw.ampr.org 2010-12-13 18:41:44 UTC ---
Created an attachment (id=4834)
 --> (https://issues.apache.org/SpamAssassin/attachment.cgi?id=4834)
Revised Patch - IPv6 checks for NetAddr::IP >= 4.000

Added check for proper version of NetAddr::IP (>= 4.000), also borrowed from
bug 6519.  However, what we really should have is a subroutine to call that
converts for us as this code is repeated in at least two if not more places.

Short of a common routine, this is commit-ready.

-- 
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 6484] Mail::SpamAssassin::Plugin::ASN - Alternative zone FORMAT enhancements.

Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6484

kd6lvw+software@kd6lvw.ampr.org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Attachment #4800|0                           |1
        is obsolete|                            |

--- Comment #1 from kd6lvw+software@kd6lvw.ampr.org 2010-09-08 19:28:52 UTC ---
Created an attachment (id=4806)
 --> (https://issues.apache.org/SpamAssassin/attachment.cgi?id=4806)
Final alpha test - I see no problems.

Here is a DIFF-patch (against SA 3.3.1) for my final version of this
enhancement to allow the Mail::SpamAssassin::Plugin::ASN module to use
alternative DNS lookup zones to obtain the ASN and network CIDR (in most cases)
for a given IPv4 address.

I do NOT solve the following issues:
1)  IPv6 addresses.  However, as these will typically be routed to a different
zone, perhaps someone should write an "asn6_lookup" directive?
2)  The fact that "_ASN_" and "_ASNCIDR_" are not available to the "report"
directive.  (Opened as a separate issue - bug # 6485 - see
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6485 for details).


There is a single minor formatting change for users of the module:  The module
no longer stores a literal "AS" as a prefix in the _ASN_ variable, and will
strip this prefix from any DNS TXT-RR that is returned as an answer to its
queries.

-- 
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.