You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Yann Nicolas <ya...@gmail.com> on 2014/02/21 06:32:43 UTC
Tomcat 7 & 8 getRemoteHost with enableLookups=true with
x-forwarded-for header gets IP instead of hostname
Hello,
I have a web application load balanced in an intranet and I need to get the
hostname of the client from the request (for audit purposes).
I have verified that the load balancer is adding the header
"x-forwarded-for" and I get the correct client IP with the
HttpServletRequest method "getRemoteAddr()". Also, I have enabled the
lookups setting to true "enableLookups" and if I connect from a client to
the server without passing through the load balancer, the hostname of the
client is correctly obtained with "getRemoteHost()".
However when I send a request from a client passing through the
load-balancer the hostname is not resolved, I get only the IP when using
the method "getRemoteHost()".
I have been looking at the source code for Tomcat 7 and Tomcat 8 and I see
that in both classes that seems to handle the x-forwarded-for header, the
hostname is never obtained from IP:
- org.apache.catalina.valves.RemoteIpValve
- org.apache.catalina.filters.RemoteIpFilter
For example in RemoteIpValve (
https://github.com/apache/tomcat/blob/trunk/java/org/apache/catalina/valves/RemoteIpValve.java),
we have:
*[...]*
if (remoteIp != null) {
request.setRemoteAddr(remoteIp);
request.setRemoteHost(remoteIp);
[...]
And the remote host is never resolved.
Of course I can put a filter in my web application to do search the
hostname from the remote IP using "java.net.InetAddress" for example but I
was wondering if a Tomcat native solution exists.
If not, is there any particular reason for this, or is it because no body
has required that feature.
Thanks,
Yann Nicolas
Re: Tomcat 7 & 8 getRemoteHost with enableLookups=true with
x-forwarded-for header gets IP instead of hostname
Posted by Yann Nicolas <ya...@gmail.com>.
Hello,
I agree, it doesn't make sense to do DNS resolution on proxy IPs.
What I mean is that I do not think it is possible to implement a logic in
Tomcat that does the reverse DNS on the IP of the client (or proxy) only if
there is no information in the x-forwarded-for header, this is done in
different sections of the request flow inside Tomcat.
It is why I think if the reverse DNS of the IP provided in x-forwarded-for
is implemented, it should be configured using a different attribute than
"enableLookups".
Regards,
Yann Nicolas
2014-02-21 10:11 GMT-06:00 Christopher Schultz <chris@christopherschultz.net
>:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> Yann,
>
> On 2/21/14, 8:53 AM, Yann Nicolas wrote:
> > Thanks a lot André and Mark,
> >
> > I understand your advice on performance degradation due to reverse
> > DNS. It makes sense to me to disable the lookups at Tomcat level
> > and search for the hostname asynchronously when storing logs (we
> > store audit in DB, then it makes even more sense do this async). I
> > will probably go for this solution.
> >
> > This is another topic, but as far as I understand (from Java7
> > javadoc), InetAddress is already implementing a cache. But it is
> > not clear to me if it is for hosname resolution (obtain the IP from
> > hostname) or reverse DNS (obtain hostname from IP). Perhaps it
> > makes sense to have our own cache of IP -> host mapping.
> >
> > Anyway, as suggested by Mark, I will create an issue in BugZilla
> > because I think it can make sense in some context to do the reverse
> > DNS lookup in Tomcat natively when using a load balancer. However I
> > am not sure if it should be better to have a new Tomcat attribute
> > for this (like enableRemoteIpLookups) instead of using the
> > attribute enableLookups, because perhaps you do not want to lookups
> > of the proxies IP but just the remoteIp (x-forwarded-for).
>
> Honestly, it seems kind of silly to do reverse-lookup on your own load
> balancers: you should know their IP addresses already and there should
> only be a few of them. What's the point in doing DNS resolution on them?
>
> - -chris
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1
> Comment: GPGTools - http://gpgtools.org
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQIcBAEBCAAGBQJTB3rDAAoJEBzwKT+lPKRYETUP/16UThlf328IzqChMez4A/EW
> +fjtzmuRuYanUatRQoxVi9Z2ckAeJO22whOMLbD16VhItgmm/YDn3wOO8wWEq5sv
> Zyb+xHlyvYpThOQ8hf/ejpx7RzqRmlr8aDZiOmyBBeGop/e84AxEk/2k0fHKRn2w
> uz+Zw8oZhhcq8UMhd6xqMk8Xs4VCRgyH6SvUo9OWARw2YkQv9Dj/zw5Pl1m3WM+U
> +Uz6NQbC8js5aUe1gZgDUUds7dFN3oLqLiuL9nY614sU8OTk4Qdwoo6i6tPKYArF
> m+C5Aya+SlfgKOgLRHyrjaWRNa+hOjldqq2kjxGhEWgtQq904hUhOuj7kWPBI/zt
> z6hdG3lmwj/heUpe/mbNXahcZ0A/UFuENT93BHVRj7ZwZHUA6Q8Qnv55Y4yFBqTd
> 2w3cZgQzGZSE0z/3qetkYd+ey2DjezLrRXHQZKb3isY3s4rlzDxNZ8dvlGY0JVdi
> CVLyzb/sbNe0v6F+EkjVIzhRn3b1iFvvsleD3pmlsWeslNsKHnDTjWDVOKdK/590
> Dyg3xGXFSAF0x3inF5S8z1QLKEem+wml/7TxW0UAC0cGAX/48DU3o1tXVa7qUYLr
> cQQUvhs/TAtpg661EQERSI/WUMpZwcyEG7djz+byLVJBppzwn1txf8ZY0H67N+1H
> wwOUN5i68TXYlp8/DTrj
> =EHo/
> -----END PGP SIGNATURE-----
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
Re: Tomcat 7 & 8 getRemoteHost with enableLookups=true with x-forwarded-for
header gets IP instead of hostname
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Yann,
On 2/21/14, 8:53 AM, Yann Nicolas wrote:
> Thanks a lot André and Mark,
>
> I understand your advice on performance degradation due to reverse
> DNS. It makes sense to me to disable the lookups at Tomcat level
> and search for the hostname asynchronously when storing logs (we
> store audit in DB, then it makes even more sense do this async). I
> will probably go for this solution.
>
> This is another topic, but as far as I understand (from Java7
> javadoc), InetAddress is already implementing a cache. But it is
> not clear to me if it is for hosname resolution (obtain the IP from
> hostname) or reverse DNS (obtain hostname from IP). Perhaps it
> makes sense to have our own cache of IP -> host mapping.
>
> Anyway, as suggested by Mark, I will create an issue in BugZilla
> because I think it can make sense in some context to do the reverse
> DNS lookup in Tomcat natively when using a load balancer. However I
> am not sure if it should be better to have a new Tomcat attribute
> for this (like enableRemoteIpLookups) instead of using the
> attribute enableLookups, because perhaps you do not want to lookups
> of the proxies IP but just the remoteIp (x-forwarded-for).
Honestly, it seems kind of silly to do reverse-lookup on your own load
balancers: you should know their IP addresses already and there should
only be a few of them. What's the point in doing DNS resolution on them?
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJTB3rDAAoJEBzwKT+lPKRYETUP/16UThlf328IzqChMez4A/EW
+fjtzmuRuYanUatRQoxVi9Z2ckAeJO22whOMLbD16VhItgmm/YDn3wOO8wWEq5sv
Zyb+xHlyvYpThOQ8hf/ejpx7RzqRmlr8aDZiOmyBBeGop/e84AxEk/2k0fHKRn2w
uz+Zw8oZhhcq8UMhd6xqMk8Xs4VCRgyH6SvUo9OWARw2YkQv9Dj/zw5Pl1m3WM+U
+Uz6NQbC8js5aUe1gZgDUUds7dFN3oLqLiuL9nY614sU8OTk4Qdwoo6i6tPKYArF
m+C5Aya+SlfgKOgLRHyrjaWRNa+hOjldqq2kjxGhEWgtQq904hUhOuj7kWPBI/zt
z6hdG3lmwj/heUpe/mbNXahcZ0A/UFuENT93BHVRj7ZwZHUA6Q8Qnv55Y4yFBqTd
2w3cZgQzGZSE0z/3qetkYd+ey2DjezLrRXHQZKb3isY3s4rlzDxNZ8dvlGY0JVdi
CVLyzb/sbNe0v6F+EkjVIzhRn3b1iFvvsleD3pmlsWeslNsKHnDTjWDVOKdK/590
Dyg3xGXFSAF0x3inF5S8z1QLKEem+wml/7TxW0UAC0cGAX/48DU3o1tXVa7qUYLr
cQQUvhs/TAtpg661EQERSI/WUMpZwcyEG7djz+byLVJBppzwn1txf8ZY0H67N+1H
wwOUN5i68TXYlp8/DTrj
=EHo/
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat 7 & 8 getRemoteHost with enableLookups=true with
x-forwarded-for header gets IP instead of hostname
Posted by Yann Nicolas <ya...@gmail.com>.
Thanks a lot André and Mark,
I understand your advice on performance degradation due to reverse DNS. It
makes sense to me to disable the lookups at Tomcat level and search for the
hostname asynchronously when storing logs (we store audit in DB, then it
makes even more sense do this async). I will probably go for this solution.
This is another topic, but as far as I understand (from Java7 javadoc),
InetAddress is already implementing a cache. But it is not clear to me if
it is for hosname resolution (obtain the IP from hostname) or reverse DNS
(obtain hostname from IP). Perhaps it makes sense to have our own cache of
IP -> host mapping.
Anyway, as suggested by Mark, I will create an issue in BugZilla because I
think it can make sense in some context to do the reverse DNS lookup in
Tomcat natively when using a load balancer. However I am not sure if it
should be better to have a new Tomcat attribute for this (like
enableRemoteIpLookups) instead of using the attribute enableLookups,
because perhaps you do not want to lookups of the proxies IP but just the
remoteIp (x-forwarded-for).
Regards,
Yann Nicolas
El feb 21, 2014 3:16 AM, "Mark Thomas" <ma...@apache.org> escribió:
> On 21/02/2014 05:32, Yann Nicolas wrote:
>
> > Of course I can put a filter in my web application to do search the
> > hostname from the remote IP using "java.net.InetAddress" for example but
> I
> > was wondering if a Tomcat native solution exists.
>
> There isn't. Please create a Bugzilla issue for this.
>
> > If not, is there any particular reason for this, or is it because no body
> > has required that feature.
>
> More an omission in the handling of x-forwarded-for I suspect.
>
> While I don't have any objections to fixing this, I do strongly
> recommend reading André's response carefully. He makes a very good point.
>
> Mark
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
Re: Tomcat 7 & 8 getRemoteHost with enableLookups=true with
x-forwarded-for header gets IP instead of hostname
Posted by Yann Nicolas <ya...@gmail.com>.
Hello,
I created an issue for this:
https://issues.apache.org/bugzilla/show_bug.cgi?id=56181
Regards,
Yann Nicolas
2014-02-21 3:15 GMT-06:00 Mark Thomas <ma...@apache.org>:
> On 21/02/2014 05:32, Yann Nicolas wrote:
>
> > Of course I can put a filter in my web application to do search the
> > hostname from the remote IP using "java.net.InetAddress" for example but
> I
> > was wondering if a Tomcat native solution exists.
>
> There isn't. Please create a Bugzilla issue for this.
>
> > If not, is there any particular reason for this, or is it because no body
> > has required that feature.
>
> More an omission in the handling of x-forwarded-for I suspect.
>
> While I don't have any objections to fixing this, I do strongly
> recommend reading André's response carefully. He makes a very good point.
>
> Mark
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
Re: Tomcat 7 & 8 getRemoteHost with enableLookups=true with x-forwarded-for
header gets IP instead of hostname
Posted by Mark Thomas <ma...@apache.org>.
On 21/02/2014 05:32, Yann Nicolas wrote:
> Of course I can put a filter in my web application to do search the
> hostname from the remote IP using "java.net.InetAddress" for example but I
> was wondering if a Tomcat native solution exists.
There isn't. Please create a Bugzilla issue for this.
> If not, is there any particular reason for this, or is it because no body
> has required that feature.
More an omission in the handling of x-forwarded-for I suspect.
While I don't have any objections to fixing this, I do strongly
recommend reading André's response carefully. He makes a very good point.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat 7 & 8 getRemoteHost with enableLookups=true with x-forwarded-for
header gets IP instead of hostname
Posted by André Warnier <aw...@ice-sa.com>.
Yann Nicolas wrote:
> Hello,
>
> I have a web application load balanced in an intranet and I need to get the
> hostname of the client from the request (for audit purposes).
>
> I have verified that the load balancer is adding the header
> "x-forwarded-for" and I get the correct client IP with the
> HttpServletRequest method "getRemoteAddr()". Also, I have enabled the
> lookups setting to true "enableLookups" and if I connect from a client to
> the server without passing through the load balancer, the hostname of the
> client is correctly obtained with "getRemoteHost()".
>
> However when I send a request from a client passing through the
> load-balancer the hostname is not resolved, I get only the IP when using
> the method "getRemoteHost()".
>
> I have been looking at the source code for Tomcat 7 and Tomcat 8 and I see
> that in both classes that seems to handle the x-forwarded-for header, the
> hostname is never obtained from IP:
> - org.apache.catalina.valves.RemoteIpValve
> - org.apache.catalina.filters.RemoteIpFilter
>
> For example in RemoteIpValve (
> https://github.com/apache/tomcat/blob/trunk/java/org/apache/catalina/valves/RemoteIpValve.java),
> we have:
>
> *[...]*
> if (remoteIp != null) {
>
> request.setRemoteAddr(remoteIp);
> request.setRemoteHost(remoteIp);
> [...]
>
> And the remote host is never resolved.
>
>
> Of course I can put a filter in my web application to do search the
> hostname from the remote IP using "java.net.InetAddress" for example but I
> was wondering if a Tomcat native solution exists.
>
> If not, is there any particular reason for this, or is it because no body
> has required that feature.
>
This is not a direct solution for you, but a general remark :
Doing a DNS lookup to obtain a hostname from an IP address can be very "expensive" and
time-consuming. This is why most webserver software disables this by default.
You probably do not want to do this "on-the-fly" for every request (*).
If you need this, it would be much better to re-process your logfiles separately off-line,
to translate these IP's into hostnames. That is what most "web statistics" programs
offer. To do this efficiently, these programs also "cache" the first response, so that
when the same IP re-occurs multiple times (as it usually does), they can translate it
without doing a DNS lookup each time.
(*) When a client requests a page from your server, that page probably contains links to
multiple additional resources that also result in more requests to your server (images,
javascript, stylesheets etc.). So each "basic" request in the end probably translates to
5 or 6 requests minimum.
If you process these requests separately and do a hostname lookup for each request, then
- the server first looks into its own "hosts" file, to see if it finds an IP->name translation
- if that doesn't work, it makes a request to its local DNS server
- if that local DNS server doesn't know, it makes a further request itself to another DNS
server
- that DNS server may not respond quickly, and then another request is made to a secondary
DNS server, etc..
And then, in the end, after losing all that time, it is very possible that no translation
could be done, because the client IP is not properly registered in "reverse DNS" (that
happens a lot with dial-up connections e.g.).
All of that can take a significant amount of time, during which your application is
waiting and not actually processing the request.
In the various webservers documentations, this is typically among the things which are
marked as "can significantly affect the performance", which is a careful way to say that
it's a killer.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org