You are viewing a plain text version of this content. The canonical link for it is here.
Posted to server-dev@james.apache.org by "Noel J. Bergman" <no...@devtech.com> on 2004/04/07 21:18:04 UTC
MxSorter and changes to DNSServer.java
I have modified getSMTPHostAddresses to call findMXRecords and then use the
Iterator from that collection. We could go back to using MxSorter (the
comment I just put in about being able to us MxSorter within findMXRecords
is wrong, since we don't ever see a Collection over which to iterate; we get
an iterator), if we make some changes to it.
The more important change in DNSServer was reverting to the previous
technique of using InetAddress.getAllByName to get the IP addresses for the
SMTP hosts, rather than the Type.A lookup. The code was failing to resolve
hosts that use CNAME on the right hand side of an MX record. That may be an
incorrect DNS configuration, but it is all too common as I have noticed over
the past couple weeks of testing the new code.
Comments? Any strong feelings either way about using MxSorter to sort on
the fly versus pre-sorting, which is what findMXRecords does? If people
want to use MxSorter, the fixes should be fairly straightforward. The
drawback is having two sections of code doing largely the same thing, but if
we end up deprecating findMXRecords, that issue goes away.
I am not sure if MxSorter is faster unless there would be a lot of records.
Even if you are using the first record, we have already got the records, and
sorting a small Collection in place is quick. MxSorter makes a linear pass
through the entire Collection, engages in object creation (at the least, it
has to call toString()) for each host matching the then leastPriorityFound,
and may create and delete entries from its working collection depending upon
the order in which they are present from the lookup call. At least that's
my take on the code.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
RE: MxSorter and changes to DNSServer.java
Posted by "Noel J. Bergman" <no...@devtech.com>.
> > I just realized that like an idiot, I just committed the
> > changes before testing them. :-( I'll be doing that
> > immediately.
> Sounds great, (except for the non-testing part ;-))
Well, at least the dogfood tastes good ...
Changing to use org.xbill.DNS.Address instead of InetAddress for the lookups
seems to be running fine so far. I haven't tried Fetchmail, though.
> > I find it bothersome that Sun's DNS service for JNDI apparently
> > ignores the TTL provided by the DNS server.
> Agreed, [the] very least they could have done was to provide
> a hook to circumvent it.
OK, this was a false alarm, and wrong. Sun's documentation is just poorly
worded, but I found a clarifying e-mail.
http://java.sun.com/j2se/1.4.2/docs/guide/net/properties.html says that
"networkaddress.cache.ttl [indicates] the caching policy for successful name
lookups from the name service." That is misleading, especially since they
later refer to sun.net.spi.nameservice.provider as specifying the name
service provider, and giving JNDI DNS as an example. However, Jayalaxmi
Hangal from Sun provided details in
http://archives.java.sun.com/cgi-bin/wa?A2=ind0402&L=jndi-interest&F=&S=&P=4
67, and clarified that the caching is not used by the JNDI DNS provider.
I don't know what Sun's provider does do in terms of caching. If we want to
replace it with dnsjava, we should be able to do so. I thought I had
remembered finding a name service stub for dnsjava on the net somewhere, but
I cannot locate it anymore. However, I did find some stub code:
http://java-house.jp/ml/archive/j-h-b/051361.html
http://java-house.jp/ml/archive/j-h-b/051940.html
It looks as if it would be fairly straightforward to plug in dnsjava, at
least with JREs that support the sun.net.spi.nameservice.provider property.
Perhaps Brian knows where there might be a ready-made adapter. I have
e-mailed him to ask.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: MxSorter and changes to DNSServer.java
Posted by Soren Hilmer <so...@tietoenator.com>.
On Tuesday 13 April 2004 03:49, Noel J. Bergman wrote:
> > I haven't looked at the org.xbill.DNS.Address but it
> > sounds like a good solution.
>
> Agreed. I'll look at making that change.
>
> > I have put it on my todo-list to go through the code and
> > weed-out InetAddress, just haven't gotten around to it yet.
>
> I just did:
>
> $ grep --recursive "InetAddress\.get.*ByName" src/java/ -l
> src/java/org/apache/james/James.java
> src/java/org/apache/james/core/AbstractJamesService.java
> src/java/org/apache/james/dnsserver/DNSServer.java
> src/java/org/apache/james/fetchmail/MessageProcessor.java
>
> src/java/org/apache/james/transport/mailets/RemoteDeliverySocketFactory.jav
>a src/java/org/apache/james/transport/mailets/RemoteDelivery.java
> src/java/org/apache/james/transport/matchers/InSpammerBlacklist.java
> src/java/org/apache/james/util/NetMatcher.java
>
> James, AbstractJamesService, and RemoteDeliverySocketFactory were all local
> stuff.
>
> I've updated DNSServer, MessageProcessor, RemoteDelivery,
> InSpammerBlacklist, and NetMatcher it use org.xbill.DNS.Address for the two
> static methods. I fully qualified the name to (a) make it easier to find,
> and (b) because it was necessary in some cases to distinguish from
> javax.mail.Address, anyway. Of course, I just realized that like an idiot,
> I just committed the changes before testing them. :-( I'll be doing that
> immediately.
Sounds great, (except for the non-testing part ;-))
>
> Another side-effect of the changes since yours and Richard's is that by
> passing an IP address to JavaMail, we eliminate its lookup, which would
> also have used InetAddress' caching.
Yes, I was aware of that "sideeffect".
>
> I find it bothersome that Sun's DNS service for JNDI apparently ignores the
> TTL provided by the DNS server.
Agreed, the Sun is absolutely NOT geared towards a mail-server solution, and
their reason for the "cache-forever" srategy (that it is made to prevent
spoofing attacks) is IMO a bit farfetched, the very least they could have
done was to provide a hook to circumvent it.
But as it is we are not too pleased with their JavaMail implementation
either ;-)
--Søren
>
> --- Noel
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
> For additional commands, e-mail: server-dev-help@james.apache.org
--
Søren Hilmer, M.Sc.
R&D manager Phone: +45 70 27 64 00
TietoEnator IT+ A/S Fax: +45 70 27 64 40
Ved Lunden 12 Direct: +45 87 46 64 57
DK-8230 Åbyhøj Email: soren.hilmer <at> tietoenator.com
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
RE: MxSorter and changes to DNSServer.java
Posted by "Noel J. Bergman" <no...@devtech.com>.
> Also, was a JIRA issue created for this change?
For the dnsjava change? Kind of bundled with another, but I've created an
explicit one for it.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: MxSorter and changes to DNSServer.java
Posted by Serge Knystautas <se...@lokitech.com>.
Vincenzo Gianferrari Pini wrote:
> Noel,
>
> after your commit I'm getting the following build error:
>
> compile:
> Compiling James Java sources
> Compiling 228 source files to C:\cvsssh\james-branch_2_1_fcs\build\classes
> C:\cvsssh\james-branch_2_1_fcs\src\java\org\apache\james\dnsserver\DNSServer.java:257: lookupRecords(org.xbill.DNS.Name,int,byte) in org.xbill.DNS.Cache cannot
> be applied to (org.xbill.DNS.Name,int,int)
> SetResponse cached = cache.lookupRecords(name, type, dnsCredibility);
> ^
> Note: Some input files use or override a deprecated API.
> Note: Recompile with -deprecation for details.
> 1 error
>
> BUILD FAILED
> file:C:/cvsssh/james-branch_2_1_fcs/build.xml:274: Compile failed; see the compiler error output for details.
>
> Vincenzo
Also, was a JIRA issue created for this change?
--
Serge Knystautas
President
Lokitech >>> software . strategy . design >> http://www.lokitech.com
p. 301.656.5501
e. sergek@lokitech.com
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
RE: MxSorter and changes to DNSServer.java
Posted by Vincenzo Gianferrari Pini <vi...@praxis.it>.
Yes, I did, and dnsjava-1.6.2.jar is in lib. But dnsjava-1.4.1.jar was still sitting there (shouldn't it have been deleted by the CVS update?), causing the problem. After removing it, the build went ok.
Vincenzo
> -----Original Message-----
> From: Noel J. Bergman [mailto:noel@devtech.com]
> Sent: martedì 13 aprile 2004 16.48
> To: James Developers List
> Subject: RE: MxSorter and changes to DNSServer.java
>
>
> > after your commit I'm getting the following build error:
>
> Did you update the whole project? There are an updated dnsjava JAR and
> build.xml.
>
> --- Noel
>
> See:
>
> http://cvs.apache.org/viewcvs.cgi/james-server/lib/?only_with_tag=branch_2_1
> _fcs
>
> http://cvs.apache.org/viewcvs.cgi/james-server/build.xml
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
> For additional commands, e-mail: server-dev-help@james.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
RE: MxSorter and changes to DNSServer.java
Posted by "Noel J. Bergman" <no...@devtech.com>.
> after your commit I'm getting the following build error:
Did you update the whole project? There are an updated dnsjava JAR and
build.xml.
--- Noel
See:
http://cvs.apache.org/viewcvs.cgi/james-server/lib/?only_with_tag=branch_2_1
_fcs
http://cvs.apache.org/viewcvs.cgi/james-server/build.xml
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
RE: MxSorter and changes to DNSServer.java
Posted by Vincenzo Gianferrari Pini <vi...@praxis.it>.
Noel,
after your commit I'm getting the following build error:
compile:
Compiling James Java sources
Compiling 228 source files to C:\cvsssh\james-branch_2_1_fcs\build\classes
C:\cvsssh\james-branch_2_1_fcs\src\java\org\apache\james\dnsserver\DNSServer.java:257: lookupRecords(org.xbill.DNS.Name,int,byte) in org.xbill.DNS.Cache cannot
be applied to (org.xbill.DNS.Name,int,int)
SetResponse cached = cache.lookupRecords(name, type, dnsCredibility);
^
Note: Some input files use or override a deprecated API.
Note: Recompile with -deprecation for details.
1 error
BUILD FAILED
file:C:/cvsssh/james-branch_2_1_fcs/build.xml:274: Compile failed; see the compiler error output for details.
Vincenzo
> -----Original Message-----
> From: Noel J. Bergman [mailto:noel@devtech.com]
> Sent: martedì 13 aprile 2004 3.50
> To: James Developers List
> Subject: RE: MxSorter and changes to DNSServer.java
>
>
> > I haven't looked at the org.xbill.DNS.Address but it
> > sounds like a good solution.
>
> Agreed. I'll look at making that change.
>
> > I have put it on my todo-list to go through the code and
> > weed-out InetAddress, just haven't gotten around to it yet.
>
> I just did:
>
> $ grep --recursive "InetAddress\.get.*ByName" src/java/ -l
> src/java/org/apache/james/James.java
> src/java/org/apache/james/core/AbstractJamesService.java
> src/java/org/apache/james/dnsserver/DNSServer.java
> src/java/org/apache/james/fetchmail/MessageProcessor.java
>
> src/java/org/apache/james/transport/mailets/RemoteDeliverySocketFactory.java
> src/java/org/apache/james/transport/mailets/RemoteDelivery.java
> src/java/org/apache/james/transport/matchers/InSpammerBlacklist.java
> src/java/org/apache/james/util/NetMatcher.java
>
> James, AbstractJamesService, and RemoteDeliverySocketFactory were all local
> stuff.
>
> I've updated DNSServer, MessageProcessor, RemoteDelivery,
> InSpammerBlacklist, and NetMatcher it use org.xbill.DNS.Address for the two
> static methods. I fully qualified the name to (a) make it easier to find,
> and (b) because it was necessary in some cases to distinguish from
> javax.mail.Address, anyway. Of course, I just realized that like an idiot,
> I just committed the changes before testing them. :-( I'll be doing that
> immediately.
>
> Another side-effect of the changes since yours and Richard's is that by
> passing an IP address to JavaMail, we eliminate its lookup, which would also
> have used InetAddress' caching.
>
> I find it bothersome that Sun's DNS service for JNDI apparently ignores the
> TTL provided by the DNS server.
>
> --- Noel
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
> For additional commands, e-mail: server-dev-help@james.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
RE: MxSorter and changes to DNSServer.java
Posted by "Noel J. Bergman" <no...@devtech.com>.
> I haven't looked at the org.xbill.DNS.Address but it
> sounds like a good solution.
Agreed. I'll look at making that change.
> I have put it on my todo-list to go through the code and
> weed-out InetAddress, just haven't gotten around to it yet.
I just did:
$ grep --recursive "InetAddress\.get.*ByName" src/java/ -l
src/java/org/apache/james/James.java
src/java/org/apache/james/core/AbstractJamesService.java
src/java/org/apache/james/dnsserver/DNSServer.java
src/java/org/apache/james/fetchmail/MessageProcessor.java
src/java/org/apache/james/transport/mailets/RemoteDeliverySocketFactory.java
src/java/org/apache/james/transport/mailets/RemoteDelivery.java
src/java/org/apache/james/transport/matchers/InSpammerBlacklist.java
src/java/org/apache/james/util/NetMatcher.java
James, AbstractJamesService, and RemoteDeliverySocketFactory were all local
stuff.
I've updated DNSServer, MessageProcessor, RemoteDelivery,
InSpammerBlacklist, and NetMatcher it use org.xbill.DNS.Address for the two
static methods. I fully qualified the name to (a) make it easier to find,
and (b) because it was necessary in some cases to distinguish from
javax.mail.Address, anyway. Of course, I just realized that like an idiot,
I just committed the changes before testing them. :-( I'll be doing that
immediately.
Another side-effect of the changes since yours and Richard's is that by
passing an IP address to JavaMail, we eliminate its lookup, which would also
have used InetAddress' caching.
I find it bothersome that Sun's DNS service for JNDI apparently ignores the
TTL provided by the DNS server.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: MxSorter and changes to DNSServer.java
Posted by Soren Hilmer <so...@tietoenator.com>.
On Friday 09 April 2004 17:12, Noel J. Bergman wrote:
> > My concern on this is that InetAddress caches successfull DNS
> > lookups forever (at least on default) and this strategy is not
> > very sound for a mailserver
>
> Good point, Søren. And that happens in InetAddress, but through
> contamination with sun.* classes. It isn't pluggable. The is a comment in
> the code that suggests that the author realizes it is a problem. Even
> replacing the default DNS provider with dnsjava (using
> sun.net.spi.nameservice.provider), would not help.
>
> However, since the use of InetAddress within DNSServer is opaque, we could
> trivially switch to using org.xbill.DNS.Address, which is a InetAddress
> clone that uses dnsjava. How does that sound?
>
I haven't looked at the org.xbill.DNS.Address but it sounds like a good
solution.
> I haven't checked the rest of the code, but InSpammerBlacklist also has
> this problem. That should probably be changed to use dnsjava, and perhaps
> JNDI in the future (portable, but more overhead). That would also allow us
> to get the TXT record, which some DNS RBLs use to provide useful
> information, e.g.,
>
> attrs = dnsContext.getAttributes(rblString, new String[] {"A", "TXT"});
>
> in JNDI-speak.
>
Yes, Noel I was aware that other places of the code might malfunction because
of this, and I have put it on my todo-list to go through the code and
weed-out InetAddress, just haven't gotten around to it yet.
-- Søren
--
Søren Hilmer, M.Sc.
R&D manager Phone: +45 70 27 64 00
TietoEnator IT+ A/S Fax: +45 70 27 64 40
Ved Lunden 12 Direct: +45 87 46 64 57
DK-8230 Åbyhøj Email: soren.hilmer <at> tietoenator.com
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
RE: MxSorter and changes to DNSServer.java
Posted by "Noel J. Bergman" <no...@devtech.com>.
> My concern on this is that InetAddress caches successfull DNS
> lookups forever (at least on default) and this strategy is not
> very sound for a mailserver
Good point, Søren. And that happens in InetAddress, but through
contamination with sun.* classes. It isn't pluggable. The is a comment in
the code that suggests that the author realizes it is a problem. Even
replacing the default DNS provider with dnsjava (using
sun.net.spi.nameservice.provider), would not help.
However, since the use of InetAddress within DNSServer is opaque, we could
trivially switch to using org.xbill.DNS.Address, which is a InetAddress
clone that uses dnsjava. How does that sound?
I haven't checked the rest of the code, but InSpammerBlacklist also has this
problem. That should probably be changed to use dnsjava, and perhaps JNDI
in the future (portable, but more overhead). That would also allow us to
get the TXT record, which some DNS RBLs use to provide useful information,
e.g.,
attrs = dnsContext.getAttributes(rblString, new String[] {"A", "TXT"});
in JNDI-speak.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: MxSorter and changes to DNSServer.java
Posted by Soren Hilmer <so...@tietoenator.com>.
Hi Noel,
My concern on this is that InetAddress caches successfull DNS lookups forever
(at least on default) and this strategy is not very sound for a mailserver,
as I happened to discover :-(
A possible (not optimal) solution is to set the security property
networkaddress.cache.ttl to something less than the maximum retry-time for an
undelivered mail, somewhere in James (RemoteDelivery ?).
--Soren
On Wednesday 07 April 2004 21:18, Noel J. Bergman wrote:
> I have modified getSMTPHostAddresses to call findMXRecords and then use the
> Iterator from that collection. We could go back to using MxSorter (the
> comment I just put in about being able to us MxSorter within findMXRecords
> is wrong, since we don't ever see a Collection over which to iterate; we
> get an iterator), if we make some changes to it.
>
> The more important change in DNSServer was reverting to the previous
> technique of using InetAddress.getAllByName to get the IP addresses for the
> SMTP hosts, rather than the Type.A lookup. The code was failing to resolve
> hosts that use CNAME on the right hand side of an MX record. That may be
> an incorrect DNS configuration, but it is all too common as I have noticed
> over the past couple weeks of testing the new code.
>
> Comments? Any strong feelings either way about using MxSorter to sort on
> the fly versus pre-sorting, which is what findMXRecords does? If people
> want to use MxSorter, the fixes should be fairly straightforward. The
> drawback is having two sections of code doing largely the same thing, but
> if we end up deprecating findMXRecords, that issue goes away.
>
> I am not sure if MxSorter is faster unless there would be a lot of records.
> Even if you are using the first record, we have already got the records,
> and sorting a small Collection in place is quick. MxSorter makes a linear
> pass through the entire Collection, engages in object creation (at the
> least, it has to call toString()) for each host matching the then
> leastPriorityFound, and may create and delete entries from its working
> collection depending upon the order in which they are present from the
> lookup call. At least that's my take on the code.
>
> --- Noel
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
> For additional commands, e-mail: server-dev-help@james.apache.org
--
Søren Hilmer, M.Sc.
R&D manager Phone: +45 70 27 64 00
TietoEnator IT+ A/S Fax: +45 70 27 64 40
Ved Lunden 12 Direct: +45 87 46 64 57
DK-8230 Åbyhøj Email: soren.hilmer <at> tietoenator.com
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
RE: MxSorter and changes to DNSServer.java
Posted by "Noel J. Bergman" <no...@devtech.com>.
> I haven't looked at this code recently, but I guess this change
> might end the reliance upon the org.xbill jar. Do you know
> offhand if it does?
No, it does not. All that the change really does is revert to the same use
we made before the change to handle multi-homed hosts. We use the DNS to
resolve the MX records, and then InetAddress to resolve the host addresses.
One of the reasons for updating to the latest dnsjava was to clear up once
and for all any licensing questions, although Serge had previously received
proper permission from Brian to include dnsjava with James.
> It is good that you caught this, Noel.
Yes, well I was watching it fail on my monitor (I keep a watch on my mail
logs in a corner of the screen), so I had some motivaton to find and fix it.
;-)
> I don't know of a good reference which tells how mailservers
> should use DNS.
RFCs 974, 2821, etc., all provide good information.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: MxSorter and changes to DNSServer.java
Posted by "Richard O. Hammer" <RO...@EarthLink.net>.
Noel J. Bergman wrote:
> The more important change in DNSServer was reverting to the previous
> technique of using InetAddress.getAllByName to get the IP addresses for the
> SMTP hosts, rather than the Type.A lookup.
I haven't looked at this code recently, but I guess this change might
end the reliance upon the org.xbill jar. Do you know offhand if it does?
> ... The code was failing to resolve
> hosts that use CNAME on the right hand side of an MX record. That may be an
> incorrect DNS configuration, but it is all too common as I have noticed over
> the past couple weeks of testing the new code.
It is good that you caught this, Noel. When I was working with this
functionality back in December, I felt unsure of the proper ways to
use DNS since I don't know of a good reference which tells how
mailservers should use DNS. Do you know of such a reference? or do we
just push ahead learning from our experience?
Rich Hammer
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org