You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Aaron Gray <aa...@gmail.com> on 2017/02/21 19:45:36 UTC
Strange URL rewrite when reverse proxy with Apache HTTP Server
I have an application server from a vendor that comes bundled with an
additional Apache Tomcat server. The webapp SelfService.war is vendor
supplied too.
Here's my problem (IP's replaced to protect the innocent):
networks:
DMZ=172.x.x.x
INTERNAL=10.x.x.x
server1 https listen = 172.1.1.1:23270
server2 https listen = 172.1.1.2:23270
F5 load balancer hostname = loadbalancer.domain.com:443
backend tomcat server = 10.1.1.1:18080
mod_proxy configuration:
ProxyPass /SelfService http://10.1.1.1:18080/SelfService
ProxyPassReverse /SelfService http://10.1.1.1:18080/SelfService
When I access these DMZ webservers which mod_proxy back to Apache Tomcat as:
https://172.1.1.1:23270/SelfService
and
https://172.1.1.2:23270/SelfService <https://172.1.1.1:23270/SelfService>
They load properly. Perfectly, every time!
When I access these DMZ webservers via the F5 load balancer (to which I
dont have access to, but the network folks configure for me), it hangs.
Eventually returns:
https://loadbalancer.domain.com:23270/SelfService
cant load.
No idea why the URL is being re-written with the ":23270".
I added static content to the server.xml on 10.1.1.1 (Tomcat) to test:
<Context docBase="/path/to/tomcat/static" path="/static" />
Then put a simple index.html in there. Accessing via the Apache Web
Servers works fine, but if you hit it with the Load Balancer it once again
adds the https://loadbalancer.domain.com:23270/static
Do you have any thoughts? Thanks so much, I have been working with this
for weeks now with no success
Re: Strange URL rewrite when reverse proxy with Apache HTTP Server
Posted by André Warnier <aw...@ice-sa.com>.
On 21.02.2017 20:45, Aaron Gray wrote:
> I have an application server from a vendor that comes bundled with an
> additional Apache Tomcat server. The webapp SelfService.war is vendor
> supplied too.
>
> Here's my problem (IP's replaced to protect the innocent):
>
> networks:
> DMZ=172.x.x.x
> INTERNAL=10.x.x.x
>
> server1 https listen = 172.1.1.1:23270
> server2 https listen = 172.1.1.2:23270
> F5 load balancer hostname = loadbalancer.domain.com:443
> backend tomcat server = 10.1.1.1:18080
>
> mod_proxy configuration:
> ProxyPass /SelfService http://10.1.1.1:18080/SelfService
> ProxyPassReverse /SelfService http://10.1.1.1:18080/SelfService
>
> When I access these DMZ webservers which mod_proxy back to Apache Tomcat as:
> https://172.1.1.1:23270/SelfService
> and
> https://172.1.1.2:23270/SelfService <https://172.1.1.1:23270/SelfService>
> They load properly. Perfectly, every time!
Ok, so there is no problem in tomcat, nor with the webapp itself.
>
> When I access these DMZ webservers via the F5 load balancer (to which I
> dont have access to, but the network folks configure for me), it hangs.
What is the URL you are using, when you do the access via the F5 load balancer ?
(just to make sure)
> Eventually returns:
> https://loadbalancer.domain.com:23270/SelfService
> cant load.
>
"Eventually" meaning that there is a delay ? how long ?
> No idea why the URL is being re-written with the ":23270".
> I added static content to the server.xml on 10.1.1.1 (Tomcat) to test:
> <Context docBase="/path/to/tomcat/static" path="/static" />
> Then put a simple index.html in there. Accessing via the Apache Web
> Servers works fine, but if you hit it with the Load Balancer it once again
> adds the https://loadbalancer.domain.com:23270/static
>
> Do you have any thoughts? Thanks so much, I have been working with this
> for weeks now with no success
>
I don't know the F5 internals, but at first sight, it sounds like 2 problems, of which the
first is a misconfiguration of the load-balancer.
Internally, it should have a configuration similar to your Apache configurations, with a
ProxyPass and a ProxyPassReverse (at least in the principle), and it should never show
this ":23270" to the caller. The URL is not really being "rewritten". It looks more like a
"leakage" of some information that the F5 should keep to itself.
It obviously replace the hostname to which it is supposed to proxy, by his own hostname
before returning this error message. But it seems to forget to also replace the port by
its own port.
The second problem, if it returns this error in the first place, seems to indicate that
there is somewhere, in addition, a communications issue between the F5 and the Apache
httpds. The F5 is obviously never getting a response from the httpds.
What is between the F5 and the httpds ? It looks like whatever it is, allows packets from
the F5 *to* a httpd but maybe not return packets *from* the httpd to the F5
(or they get wrongly routed back).
I have seen this kind of issue before, with servers hosted "in the cloud".
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Strange URL rewrite when reverse proxy with Apache HTTP Server
Posted by "André Warnier (tomcat)" <aw...@ice-sa.com>.
On 23.02.2017 19:10, Aaron Gray wrote:
> Okay, guys, please accept my most sincere apology on this thing. I just
> figured 1 thing out.
>
> browser -> f5:443 -> httpd (23270, https) -> tomcat (http)
> using the /static works! (see below)
>
> If /static/index.html wasnt specified, then it hangs, then comes back as
> https://loadbalancer.domain.com:23270/static
> But if you go https://loadbalancer.domain.com/static/index.html it works!!!
>
> DirectoryIndex index.html
> is present. because if you just go to https://webserver1:23270/static
> it works fine, no need for the extra /index.html
>
Something vague in my mind tells me that the problem is probably right there, in the
Redirect response, but it's quite late here in Europe now, and I can't quite figure it out
anymore..
> not sure why that is different there.
>
> I only found this out using curl and just happened to think of it.
> So you see the difference, from the tomcat access log, in the two
> variations.
>
> 172.18.160.66 - - [23/Feb/2017:10:00:48 -0800] "GET /static HTTP/1.1" 302 -
> 172.18.160.66 - - [23/Feb/2017:10:03:04 -0800] "GET /static/index.html
> HTTP/1.1" 200 32
You must be very selectively filtering this log, because :
- the first response above is a Redirect response, normal because of the DirectoryIndex
- the second response /could/ be the follow-up request that the browser makes after this
Redirect, /except/ that it seems to happen almost 3 minutes later, which would mean either
a /very/ slow connection, or a /very/ slow browser.
>
> Still /SelfService wont work though
> I cant think of a way to do /SelfService/something...
neither should you need to, unless this is also affected by the Redirect syndrom.
> Since it is a web app and i am no developer and its all from the vendor
> (BMC Software) and it never shows naything other than /SelfService in the
> URL. I dont think its a frame, but once again i dont know these things. I
> know they told me it is HTML5, as it was using Silverlight back in the
> older release.
None of this should matter, really.
What about these Apache and tomcat *error logs* ? what do they tell you, when the initial
problem occurs ?
>
>
> On Thu, Feb 23, 2017 at 9:22 AM, Aaron Gray <aa...@gmail.com> wrote:
>
>> Working with my F5 guy, we had an idea, since 80/tcp and 443/tcp were
>> already open to the VIP on the F5, we simply turned of 80 -> 443 redirect
>> on the F5, and then configured the F5 to use the non-HTTP port in Apache
>> HTTP Server (the two backend servers). So its HTTP the entire way through,
>> and the same issue persisted.
>>
>> browser -> F5(http) -> httpd(http port) -> tomcat(http port, nonssl)
>>
>> So i think i can rule out any possibility that the fact I was terminating
>> SSL in Apache HTTP Server then proxying clear-text back as a reason. It
>> worked fine like I said if you just go direct to the HTTP Server and slip
>> the F5. And F5 works landing the /healthcheck.html (HTTP KeepAlive in the
>> root, which isnt proxied), so all good there. Just cant /SelfService
>> (configured w/ Proxy)
>>
>> thanks for your support here though.
>>
>> On Thu, Feb 23, 2017 at 9:12 AM, Aaron Gray <aa...@gmail.com> wrote:
>>
>>> SSLProxyEngine On
>>> Was already turned on this entire time inside the ssl.conf (I include it)
>>> VirtualHost section.
>>>
>>> I am debating turning on HTTPS in Tomcat on the backend 10.x.x.x app
>>> server, and then HTTPS the whole way through and see if that makes any
>>> difference. I may need to request a new firewall to be opened, which is
>>> not able to open until 3/2. Gonna see.
>>>
>>>
>>> On Thu, Feb 23, 2017 at 8:22 AM, Antonio S. Cofi�o <co...@gmail.com>
>>> wrote:
>>>
>>>>
>>>> On 23/02/17 12:43, Andr� Warnier (tomcat) wrote:
>>>>
>>>>> On 22.02.2017 19:22, Aaron Gray wrote:
>>>>>
>>>>>> So this is interesting.
>>>>>>
>>>>>> So from HTTP server #1 (172.1.1.1 example) I hit:
>>>>>> https://172.1.1.1:23270/static and I see this in the HTTP log:
>>>>>> 172.1.1.1 - - [22/Feb/2017:10:14:48 -0800] "GET /static/ HTTP/1.1" 200
>>>>>> 32
>>>>>> I see this in the Tomcat log:
>>>>>> 172.1.1.1 - - [22/Feb/2017:10:14:48 -0800] "GET /static/ HTTP/1.1" 200
>>>>>> 32
>>>>>> ((( Yes I know they look identical, but the logformat is the same, so
>>>>>> it
>>>>>> comes out looking the same, but they are two diff log files totally
>>>>>> )))
>>>>>>
>>>>>> I then hit the https://loadbalancer.domain.com/static from my Win10
>>>>>> laptop
>>>>>> (10.1.1.2 example)
>>>>>> and I see this in the HTTP server #1 log, but NOTHING in the tomcat
>>>>>> access
>>>>>> log.
>>>>>> 10.1.1.2 - - [22/Feb/2017:10:20:02 -0800] "-" 408 -
>>>>>>
>>>>>> So 408. Timeout. Hmm... Why? We KNOW that it can connect from http
>>>>>> ->
>>>>>> tomcat:18080 perfectly. So is it a timeout BACK to the F5? That i
>>>>>> dont
>>>>>> know yet.
>>>>>>
>>>>>
>>>>> It looks as if, in that case, httpd is trying to connect to tomcat, but
>>>>> - either it /is/ connecting, and sending the request, but tomcat never
>>>>> answers
>>>>> - or it is connecting to something else, which isn't tomcat
>>>>> (but which accepts the request and doesn't answer either)
>>>>>
>>>>> I am not *sure*, but I believe, that if httpd was trying to connect to
>>>>> something that isn't tomcat, and is not listening on port 18080, it would
>>>>> get a different error, and a different logfile message. So it would seem
>>>>> that at least the TCP connection works.
>>>>>
>>>>
>>>>
>>>> Is the F5 loadbalancer configured to make reverse proxy to a SSL http?
>>>> The directive SSLProxyEngine onthe F5 loadbalencer has to be enabled.
>>>> http://httpd.apache.org/docs/current/mod/mod_ssl.html#sslproxyengine
>>>>
>>>> If the loadbalancer it's delegating the SSL work to the server1
>>>> server1 https listen = 172.1.1.1:23270
>>>>
>>>> then the F5 could be connecting as http regular TCP connection to a SSL
>>>> TCP one and therefor the SSL handshake it's not happening.
>>>>
>>>> Antonio
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> Now there is still a puzzle there : when you connect as
>>>>> browser --> httpd --> tomcat
>>>>> then everything works : httpd connects to tomcat, sends a request, gets
>>>>> a response, and returns it to the browser.
>>>>>
>>>>> But when you do the same as :
>>>>> browser --> F5 --> httpd --> tomcat
>>>>> it "does not work", and the problem seems to be at httpd --> tomat
>>>>> Why would it make a difference in httpd --> tomcat, if it is either the
>>>>> browser or the F5 which sends the request to httpd ?
>>>>> the httpd --> tomcat connection should be the same, no matter what.
>>>>>
>>>>> Hmm, for now as puzzled as you are.
>>>>> Maybe we have a quantum phenomenon here, where the mere fact of you
>>>>> looking at the logfile, changes what happened beforehand. Just kidding..
>>>>>
>>>>> Informational note : the Apache access log line, is written when Apache
>>>>> *has finished* processing the request (including the possible round-trip
>>>>> through tomcat) and has sent the response to the client. (That's because,
>>>>> for example, it can also write the time spent, and the size of the
>>>>> response; so it has to know those before it can write the log line).
>>>>> I presume that it is the same for tomcat.
>>>>> Just mentioning this, so that you would maybe have a second look at the
>>>>> tomcat access log, after a few minutes, to make sure that there really is
>>>>> no corresponding line there.
>>>>> It could be that tomcat actually /is/ receiving the request, and
>>>>> processing it, but that it takes a long time, and that httpd tires of
>>>>> waiting and already times out in the meantime. In such a case, you would
>>>>> probably see an error some time later in the tomcat error log, when it is
>>>>> trying to eventually return the response, and finds a closed connection
>>>>> instead.
>>>>> Now again, why it would do that in one case and not the other, is still
>>>>> a mystery.
>>>>>
>>>>> Another note : in the "timeout" log line above, httpd is not showing
>>>>> the request URL which triggered this timeout.
>>>>> a) it may be showing some additional information in the Apache error log
>>>>> b) these exists an httpd add-on module (mod_log_forensic ?) which could
>>>>> provide more details in the log, about what happens precisely
>>>>>
>>>>>
>>>>>
>>>>>> On Tue, Feb 21, 2017 at 3:39 PM, Andr� Warnier (tomcat) <aw@ice-sa.com
>>>>>>>
>>>>>> wrote:
>>>>>>
>>>>>> On 21.02.2017 23:28, Aaron Gray wrote:
>>>>>>>
>>>>>>> Antonio: The Tomcat server has no knowledge of the F5, or that it is
>>>>>>>> being
>>>>>>>> fronted by an Apache HTTP Server. I do SSL termination in Apache
>>>>>>>> HTTP
>>>>>>>> Server, and clear-text from HTTP to Tomcat.
>>>>>>>> My redirect port for the normal HTTP listen in Tomcat is commented
>>>>>>>> out.
>>>>>>>> <Connector port="18080" protocol="HTTP/1.1"
>>>>>>>> connectionTimeout="20000" />
>>>>>>>> <!-- A "Connector" using the shared thread pool-->
>>>>>>>> <!--
>>>>>>>> <Connector executor="tomcatThreadPool"
>>>>>>>> port="8080" protocol="HTTP/1.1"
>>>>>>>> connectionTimeout="20000"
>>>>>>>> redirectPort="8443" />
>>>>>>>> -->
>>>>>>>>
>>>>>>>> Andre:
>>>>>>>> The URL I am using is https://loadbalancer.domain.com
>>>>>>>> It is listening on port 80 and 443, if you hit 80, internally it
>>>>>>>> redirects
>>>>>>>> you to 443. No SSL cert on the F5 load balancer. It simply sends
>>>>>>>> the
>>>>>>>> traffic to one of the two HTTP servers (round-robin, also tried
>>>>>>>> persistence, no difference). The HTTP server is listening only for
>>>>>>>> HTTPS
>>>>>>>> on 23270/tcp.
>>>>>>>> When I access these DMZ webservers via the F5 load balancer (to
>>>>>>>> which I
>>>>>>>> dont have access to, but the network folks configure for me), it
>>>>>>>> hangs.
>>>>>>>> Eventually returns: server1 https listen = 172.1.1.1:23270
>>>>>>>>
>>>>>>>> https://loadbalancer.domain.com:23270/SelfService
>>>>>>>> cant load.
>>>>>>>>
>>>>>>>> No idea why the URL is being re-written with the ":23270".
>>>>>>>> I added static content to the server.xml on 10.1.1.1 (Tomcat) to
>>>>>>>> test:
>>>>>>>> <Context docBase="/path/to/tomcat/static" path="/static" />
>>>>>>>> Then put a simple index.html in there. Accessing via the Apache Web
>>>>>>>> Servers works fine, but if you hit it with the Load Balancer it once
>>>>>>>> again
>>>>>>>> adds the https://loadbalancer.domain.com:23270/static
>>>>>>>>
>>>>>>>> Hitting https://loadbalancer.domain.com
>>>>>>>> I see my "Hello world!" which is all that is in index.html. This is
>>>>>>>> the
>>>>>>>> DocumentRoot of HTTP, and *not* proxied over at this time.
>>>>>>>>
>>>>>>>>
>>>>>>> So in this case, there is no delay, and you get the Apache
>>>>>>> httpd-hosted
>>>>>>> "index.html" containing "Hello World. Right ?
>>>>>>>
>>>>>>> Only
>>>>>>>
>>>>>>> /SelfService and /static are proxied
>>>>>>>> /static just being my test of static content, but still served up by
>>>>>>>> Tomcat..
>>>>>>>>
>>>>>>>> It's exactly 30 seconds before the page cannot be loaded when trying
>>>>>>>> anything proxied to Tomcat, but also accessed via the F5 load
>>>>>>>> balancer.
>>>>>>>> Not sure where the 30 seconds comes from; perhaps a load balancer
>>>>>>>> time
>>>>>>>> out,
>>>>>>>> as I dont see a "30" in my httpd configurations or my tomcat
>>>>>>>> server.xml
>>>>>>>>
>>>>>>>>
>>>>>>>> You can certainly look at the Apache httpd logs, and the tomcat
>>>>>>> logs, to
>>>>>>> see if you get a request or not.
>>>>>>> In Apache httpd, you can set the loglevel individually for mod_proxy
>>>>>>> (if
>>>>>>> you are running v 2.4), and it should show something if it gets this
>>>>>>> request and forwards it to tomcat.
>>>>>>> In tomcat, you can either enable an access log (which will show if it
>>>>>>> receives this request), or you could temporarily remove/rename the
>>>>>>> /static
>>>>>>> webapp. This way, it should trigger an error "not found" which you
>>>>>>> would
>>>>>>> also see in the error log.
>>>>>>>
>>>>>>> There should be nothing between them to hinder it. We have many load
>>>>>>>
>>>>>>>> balancers and this one specifically you dont need to open any
>>>>>>>> firewall
>>>>>>>> requests for the specific networks the HTTP servers are on. I did
>>>>>>>> have to
>>>>>>>> get the firewall opened up to allow me to hit
>>>>>>>> https://loadbalancer.domain.com because the VIP for "
>>>>>>>> loadbalancer.domain.com"
>>>>>>>> is in the DMZ, and my Desktop & VPN networks cannot hit it on 80/443
>>>>>>>> without opening holes. But beyond that, any connection from the F5
>>>>>>>> to the
>>>>>>>> HTTP Server should be 100% open bi-directional, since same subnet.
>>>>>>>>
>>>>>>>>
>>>>>>>> But something isn't working, otherwise you would not be asking.
>>>>>>> So,
>>>>>>>
>>>>>>> a) hitting the tomcat webapps through httpd seems to be working fine
>>>>>>> (browser -> httpd:23270 -> tomcat:18080 -> webapp or static)
>>>>>>>
>>>>>>> b) hitting a non-proxied-to-tomcat resource of httpd seems to work
>>>>>>> fine
>>>>>>> too, even through the F5
>>>>>>> (browser -> F5:443 -> httpd:23270 -> html page)
>>>>>>>
>>>>>>> c) it is only when you do :
>>>>>>> (browser -> F5:443 -> httpd:23270 -> tomcat:18080 -> webapp or
>>>>>>> static)
>>>>>>> that you see this issue
>>>>>>>
>>>>>>> It would really help if you looked in the logs of both httpd and
>>>>>>> tomcat,
>>>>>>> and checked for differences betweens cases a, b and c above.
>>>>>>>
>>>>>>> I believe that the F5 message with the port 23270 is a minor issue, of
>>>>>>> information disclosure by the F5, that it should not disclose.
>>>>>>>
>>>>>>> But the reason why it returns this error is obviously that in that
>>>>>>> case,
>>>>>>> it does not get a response from his request to httpd.
>>>>>>> The reason for this response not coming back to the F5 (in case c
>>>>>>> only),
>>>>>>> can be due to either httpd or tomcat. But F5 doesn't know about
>>>>>>> tomcat. So
>>>>>>> for the F5, it is httpd which is not responding. Thus,
>>>>>>> - either httpd is never getting the request from the F5 (unlikely,
>>>>>>> because
>>>>>>> in b above it gets it and responds)
>>>>>>> - or httpd is getting the request from the F5, but not forwarding it
>>>>>>> to
>>>>>>> tomcat, but also not returning an immediate error response to F5
>>>>>>> (which
>>>>>>> seems also unlikely, because of a and b)
>>>>>>> - or httpd is getting the request, forwarding it to tomcat, but not
>>>>>>> getting a response from tomcat. So
>>>>>>> - either tomcat is never getting the request from httpd (but in a,
>>>>>>> it
>>>>>>> gets it)
>>>>>>> - or tomcat is getting the request from httpd, but not responding
>>>>>>> (but
>>>>>>> in a, it does)
>>>>>>> - or tomcat is getting the request and responding, but the response
>>>>>>> never gets back to httpd (but in a, it does)
>>>>>>>
>>>>>>> So if a and b and c are all accurate, there is something apparently
>>>>>>> illogical happening.
>>>>>>> This would lead to the conclusion that a and b and c cannot all be
>>>>>>> accurate.
>>>>>>>
>>>>>>> The logs.. ?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Feb 21, 2017 at 2:05 PM, Antonio S. Cofino <cofinoa@gmail.com
>>>>>>>>
>>>>>>>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Aaron, on tomcat instances change the redirectPort attributte on the
>>>>>>>> http
>>>>>>>>
>>>>>>>>> conect� to the loabbalancer's port 443
>>>>>>>>>
>>>>>>>>> My guess is that your webapp has restriction rule requesting SSL con
>>>>>>>>> fidntial channel. Therefore the non-confidential to the 18080 port
>>>>>>>>> from
>>>>>>>>> the
>>>>>>>>> balancer are redirected to the 23270 port, but it should be 443.
>>>>>>>>>
>>>>>>>>> Antonio
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> El 21/2/2017 19:46, "Aaron Gray" <aa...@gmail.com> escribi�:
>>>>>>>>>
>>>>>>>>> I have an application server from a vendor that comes bundled with
>>>>>>>>> an
>>>>>>>>> additional Apache Tomcat server. The webapp SelfService.war is
>>>>>>>>> vendor
>>>>>>>>> supplied too.
>>>>>>>>>
>>>>>>>>> Here's my problem (IP's replaced to protect the innocent):
>>>>>>>>>
>>>>>>>>> networks:
>>>>>>>>> DMZ=172.x.x.x
>>>>>>>>> INTERNAL=10.x.x.x
>>>>>>>>>
>>>>>>>>> server1 https listen = 172.1.1.1:23270
>>>>>>>>> server2 https listen = 172.1.1.2:23270
>>>>>>>>> F5 load balancer hostname = loadbalancer.domain.com:443
>>>>>>>>> backend tomcat server = 10.1.1.1:18080
>>>>>>>>>
>>>>>>>>> mod_proxy configuration:
>>>>>>>>> ProxyPass /SelfService http://10.1.1.1:18080/SelfService
>>>>>>>>> ProxyPassReverse /SelfService http://10.1.1.1:18080/SelfService
>>>>>>>>>
>>>>>>>>> When I access these DMZ webservers which mod_proxy back to Apache
>>>>>>>>> Tomcat
>>>>>>>>> as:
>>>>>>>>> https://172.1.1.1:23270/SelfService
>>>>>>>>> and
>>>>>>>>> https://172.1.1.2:23270/SelfService <https://172.1.1.1:23270/SelfS
>>>>>>>>> ervice
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> They load properly. Perfectly, every time!
>>>>>>>>>
>>>>>>>>> When I access these DMZ webservers via the F5 load balancer (to
>>>>>>>>> which I
>>>>>>>>> dont have access to, but the network folks configure for me), it
>>>>>>>>> hangs.
>>>>>>>>> Eventually returns:
>>>>>>>>> https://loadbalancer.domain.com:23270/SelfService
>>>>>>>>> cant load.
>>>>>>>>>
>>>>>>>>> No idea why the URL is being re-written with the ":23270".
>>>>>>>>> I added static content to the server.xml on 10.1.1.1 (Tomcat) to
>>>>>>>>> test:
>>>>>>>>> <Context docBase="/path/to/tomcat/static" path="/static" />
>>>>>>>>> Then put a simple index.html in there. Accessing via the Apache Web
>>>>>>>>> Servers works fine, but if you hit it with the Load Balancer it once
>>>>>>>>> again
>>>>>>>>> adds the https://loadbalancer.domain.com:23270/static
>>>>>>>>>
>>>>>>>>> Do you have any thoughts? Thanks so much, I have been working with
>>>>>>>>> this
>>>>>>>>> for weeks now with no success
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>>>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>>>
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>>
>>>>
>>>
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Strange URL rewrite when reverse proxy with Apache HTTP Server
Posted by Aaron Gray <aa...@gmail.com>.
I found the old notes from years ago when we were on tomcat 6 (included
with the BMC software app) and did the same thing, renaming their "tomcat"
folder with the extracted 7.0.75 and did *.war (more .war's in this new
release and the instructions below state) and tweaks to server.xml etc
It works now. 100% W O R K S!
I dont know why yet, and there's so many things I could test (like
downloading the exact same release of tomcat they used and see if my
vanilla one works, and theirs doesnt. Or check release notes for
subsequent releases, etc.) but I will put that on the vendor.
Pretty crazy that after all this, it was at least in testing, solved like
this. Man! wow.
**** NOTES ****
1. Stop the Control-M Web Server.
2. Rename folder tomcat to tomcat.orig, folders path:
a. Windows: %EM_HOME%/emweb/tomcat
b. UNIX: $EM_HOME/etc/emweb/tomcat
3. Download tomcat 6.0.41 Binaries from http://tomcat.apache.org/
download-60.cgi#6.0.41 (Binary distribution Core, ZIP or tar.gz file)
4. Extract the downloaded file to "emweb" folder (%EM_HOME%/emweb or
$EM_HOME/etc/emweb/tomcat)
5. Rename the root folder name from "apache-tomcat-6.0.41" to "tomcat"
6. Copy the following files from "tomcat.orig" folder to "tomcat" folder
using their relative path:
a. tomcat.orig/webapps/SelfService.war
b. tomcat.orig/webapps/WorkloadChangeManager.war
c. tomcat.orig/webapps/bim.war
d. tomcat.orig/conf/server.xml (overwrite existing)
e. tomcat.orig/webapps/ROOT/index.html (overwrite existing)
f. If SSL is enable on Tomcat, please copy the certificate located at
tomcat.orig/conf
7. Edit the copied tomcat/conf/server.xml xml file and add the following
lines where other listeners are defined:
<!-- Prevent memory leaks due to use of particular java/javax APIs-->
<Listener className="org.apache.catalina.core.JreMemoryLeakPreventionListener"
/>
For example, in the default "server.xml", the lines are as follows:
<!--APR library loader. Documentation at /docs/apr.html -->
<Listener className="org.apache.catalina.core.AprLifecycleListener"
SSLEngine="on" />
<!--Initialize Jasper prior to webapps are loaded. Documentation at
/docs/jasper-howto.html -->
<Listener className="org.apache.catalina.core.JasperListener" />
<!-- JMX Support for the Tomcat server. Documentation at
/docs/non-existent.html -->
<Listener className="org.apache.catalina.mbeans.ServerLifecycleListener" />
<Listener className="org.apache.catalina.mbeans.
GlobalResourcesLifecycleListener" />
The new section should look like:
<!--APR library loader. Documentation at /docs/apr.html -->
<Listener className="org.apache.catalina.core.AprLifecycleListener"
SSLEngine="on" />
<!--Initialize Jasper prior to webapps are loaded. Documentation at
/docs/jasper-howto.html -->
<Listener className="org.apache.catalina.core.JasperListener" />
<!-- JMX Support for the Tomcat server. Documentation at
/docs/non-existent.html -->
<Listener className="org.apache.catalina.mbeans.ServerLifecycleListener" />
<Listener className="org.apache.catalina.mbeans.
GlobalResourcesLifecycleListener" />
<!-- Prevent memory leaks due to use of particular java/javax APIs-->
<Listener className="org.apache.catalina.core.JreMemoryLeakPreventionListener"
/>
8. Start the Control-M Web Server
On Fri, Feb 24, 2017 at 8:29 AM, Aaron Gray <aa...@gmail.com> wrote:
> Andre, that is all very educational and I feel even when this is resolved,
> that I have learned a good amount here; so I thank you for everything.
>
> I have an update. I decided to shutdown the vendor included/supplied
> 7.0.50 release of Tomcat. I extracted vanilla 7.0.75 tomcat, updated
> server.xml to change port from 8080 to 18080, and added the static line
> (created a new /static location like:
> /app/home/agray/apache-tomcat-7.0.75/static
> Started that tomcat
>
> https://loadbalancer.domain.com/static
> loads PERFECTLY! Whaaaatttt? Now I guess I could be investigating the
> differences between these two environments. I do remember though they had
> a document that I followed a long time ago when the release of Tomcat they
> included had a major vulnerability in it (years ago) to copy their specific
> .war and changes in to a vanilla Tomcat and deploy it there. I think I
> want to do that. I have escalated to their developers, since I can prove a
> vanilla tomcat can serve up my static content through the browser -> F5 ->
> apache http proxy server -> app server yet their included one cant
> without 302 and adding the :port and having to remove it, etc.
>
> Wild. I'll update soon.
> BTW, hope you got some sleep after last night, but want to thank you again.
>
> On Thu, Feb 23, 2017 at 4:00 PM, André Warnier (tomcat) <aw...@ice-sa.com>
> wrote:
>
>> To avoid getting totally confused, you may want to read this explanation
>> first :
>> https://wiki.apache.org/httpd/DirectoryListings
>>
>> In other words, when you send this request from the browser :
>> https://loadbalancer.domain.com/SelfService
>>
>> The first response that /should/ come back, would be a "re-direct"
>> response from the (final) server, with a special HTTP header added :
>> Location: https://loadbalancer.domain.com/SelfService/
>>
>> Then your browser should re-issue the request automatically, this time to
>> :
>> https://loadbalancer.domain.com/SelfService/
>> (with the trailing "/" added)
>>
>> Then, in a second step, what should happen is :
>>
>> Your browser indicates a URL, which on the server resolves to a directory.
>> The server /may/ be configured in such a way, that for this kind of
>> request, it returns an auto-generated list of all the files in that
>> directory, automatically.
>> Or it may be configured to return instead, a specific html page.
>> (That what you do in Apache, with the DirectoryIndex directive)
>> And if the server is not configured at all for that, it would probably
>> return a response "Forbidden", for security reasons (you probably don't
>> want someone to be able to list the files in that directory, unless you
>> specifically configured the server to allow that).
>>
>> Or, the server may be configured so that for such a request, it defaults
>> to running some standard servlet which is part of the application, and
>> which would return the application starting page.
>> That would probably be the case in your "/SelfService" tomcat webapp.
>>
>> So, the 302 responses which you get in some cases, are probably normal,
>> as explained by all the above.
>> What is definitely /not/ normal, is that port that you get either in
>> these redirect responses, or in the error messages. Because that means
>> that, somewhere in the chain, one of the elements is not doing its job
>> properly when proxying.
>> The point of the ProxyPassReverse directive, is precisely that : ensure
>> that if the "proxied to" server returns a re-direct response, the
>> "Location:" header that is in that redirect response would be properly
>> edited before returning that redirect response back up the chain, in such a
>> way that the ultimate client would not try next, to access a URL that leads
>> nowhere.
>>
>> And now it's really late..
>>
>>
>>
>> On 24.02.2017 00:06, André Warnier (tomcat) wrote:
>>
>>> On 23.02.2017 21:17, Aaron Gray wrote:
>>>
>>>> Another weird thing...
>>>>
>>>> If you go to:
>>>> https://loadbalancer.domain.com/SelfService
>>>> If I am viewing the headers immedaitely I see 302 not found.
>>>>
>>>
>>> 302 is not "not found". It is "Found", and it is a Redirect response.
>>> See : https://en.wikipedia.org/wiki/HTTP_302
>>> for a good explanation.
>>> (But stop after the second paragraph, otherwise you'll get confused
>>> again..)
>>>
>>> Then I wait
>>>
>>>> the 30 sec for it to totally fail and change the URL in the browser to
>>>> https://loadbalancer.domain.com:232700/SelfService
>>>>
>>>> If you just remove the :232700 and hit Enter, it then loads properly.
>>>> See
>>>> immedaitly 200 OK and it is fine. Don't navigate anywhere else like to
>>>> https://loadbalancer.domain.com/static or else you'll have to repeat
>>>> the
>>>> same thing as above with the :232700 and removing it, hit Enter.
>>>>
>>>> Weird.
>>>>
>>>> Of course none of this is an issue if you dont hit it with the F5 Load
>>>> Balancer to start.
>>>>
>>>
>>> Yes. There seems to be something weird *when a Redirect response is
>>> being sent back by
>>> tomcat*, and has to go back through the load balancer.
>>> And it may be linked to the fact that the load-balancer does not seem to
>>> remove this
>>> :23270 port which eventually reaches the browser.
>>>
>>>
>>>
>>>
>>>> On Thu, Feb 23, 2017 at 10:24 AM, Aaron Gray <aa...@gmail.com>
>>>> wrote:
>>>>
>>>> I randomly tried https://loadbalancer.domain.com/SelfService/index.html
>>>>>
>>>>> It works perfectly. No issues. I have no idea why its needed to add
>>>>> on
>>>>> the /index.html (i'm lucky this exists in the tomcat .war, as it was a
>>>>> guess). This is visually not appealing, but workable. Maybe I can do
>>>>> some
>>>>> url rewrite or something, IF this cant be fixed normally?
>>>>>
>>>>> On Thu, Feb 23, 2017 at 10:10 AM, Aaron Gray <aa...@gmail.com>
>>>>> wrote:
>>>>>
>>>>> Okay, guys, please accept my most sincere apology on this thing. I
>>>>>> just
>>>>>> figured 1 thing out.
>>>>>>
>>>>>> browser -> f5:443 -> httpd (23270, https) -> tomcat (http)
>>>>>> using the /static works! (see below)
>>>>>>
>>>>>> If /static/index.html wasnt specified, then it hangs, then comes back
>>>>>> as
>>>>>> https://loadbalancer.domain.com:23270/static
>>>>>> But if you go https://loadbalancer.domain.com/static/index.html it
>>>>>> works!!!
>>>>>>
>>>>>> DirectoryIndex index.html
>>>>>> is present. because if you just go to https://webserver1:23270/stati
>>>>>> c
>>>>>> it works fine, no need for the extra /index.html
>>>>>>
>>>>>> not sure why that is different there.
>>>>>>
>>>>>> I only found this out using curl and just happened to think of it.
>>>>>> So you see the difference, from the tomcat access log, in the two
>>>>>> variations.
>>>>>>
>>>>>> 172.18.160.66 - - [23/Feb/2017:10:00:48 -0800] "GET /static HTTP/1.1"
>>>>>> 302
>>>>>> -
>>>>>> 172.18.160.66 - - [23/Feb/2017:10:03:04 -0800] "GET /static/index.html
>>>>>> HTTP/1.1" 200 32
>>>>>>
>>>>>> Still /SelfService wont work though
>>>>>> I cant think of a way to do /SelfService/something...
>>>>>> Since it is a web app and i am no developer and its all from the
>>>>>> vendor
>>>>>> (BMC Software) and it never shows naything other than /SelfService in
>>>>>> the
>>>>>> URL. I dont think its a frame, but once again i dont know these
>>>>>> things. I
>>>>>> know they told me it is HTML5, as it was using Silverlight back in the
>>>>>> older release.
>>>>>>
>>>>>>
>>>>>> On Thu, Feb 23, 2017 at 9:22 AM, Aaron Gray <aa...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>> Working with my F5 guy, we had an idea, since 80/tcp and 443/tcp were
>>>>>>> already open to the VIP on the F5, we simply turned of 80 -> 443
>>>>>>> redirect
>>>>>>> on the F5, and then configured the F5 to use the non-HTTP port in
>>>>>>> Apache
>>>>>>> HTTP Server (the two backend servers). So its HTTP the entire way
>>>>>>> through,
>>>>>>> and the same issue persisted.
>>>>>>>
>>>>>>> browser -> F5(http) -> httpd(http port) -> tomcat(http port, nonssl)
>>>>>>>
>>>>>>> So i think i can rule out any possibility that the fact I was
>>>>>>> terminating SSL in Apache HTTP Server then proxying clear-text back
>>>>>>> as a
>>>>>>> reason. It worked fine like I said if you just go direct to the HTTP
>>>>>>> Server and slip the F5. And F5 works landing the /healthcheck.html
>>>>>>> (HTTP
>>>>>>> KeepAlive in the root, which isnt proxied), so all good there. Just
>>>>>>> cant
>>>>>>> /SelfService (configured w/ Proxy)
>>>>>>>
>>>>>>> thanks for your support here though.
>>>>>>>
>>>>>>> On Thu, Feb 23, 2017 at 9:12 AM, Aaron Gray <aa...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>> SSLProxyEngine On
>>>>>>>> Was already turned on this entire time inside the ssl.conf (I
>>>>>>>> include
>>>>>>>> it) VirtualHost section.
>>>>>>>>
>>>>>>>> I am debating turning on HTTPS in Tomcat on the backend 10.x.x.x app
>>>>>>>> server, and then HTTPS the whole way through and see if that makes
>>>>>>>> any
>>>>>>>> difference. I may need to request a new firewall to be opened,
>>>>>>>> which is
>>>>>>>> not able to open until 3/2. Gonna see.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Feb 23, 2017 at 8:22 AM, Antonio S. Cofiño <
>>>>>>>> cofinoa@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>> On 23/02/17 12:43, André Warnier (tomcat) wrote:
>>>>>>>>>
>>>>>>>>> On 22.02.2017 19:22, Aaron Gray wrote:
>>>>>>>>>>
>>>>>>>>>> So this is interesting.
>>>>>>>>>>>
>>>>>>>>>>> So from HTTP server #1 (172.1.1.1 example) I hit:
>>>>>>>>>>> https://172.1.1.1:23270/static and I see this in the HTTP log:
>>>>>>>>>>> 172.1.1.1 - - [22/Feb/2017:10:14:48 -0800] "GET /static/
>>>>>>>>>>> HTTP/1.1"
>>>>>>>>>>> 200 32
>>>>>>>>>>> I see this in the Tomcat log:
>>>>>>>>>>> 172.1.1.1 - - [22/Feb/2017:10:14:48 -0800] "GET /static/
>>>>>>>>>>> HTTP/1.1"
>>>>>>>>>>> 200 32
>>>>>>>>>>> ((( Yes I know they look identical, but the logformat is the
>>>>>>>>>>> same,
>>>>>>>>>>> so it
>>>>>>>>>>> comes out looking the same, but they are two diff log files
>>>>>>>>>>> totally
>>>>>>>>>>> )))
>>>>>>>>>>>
>>>>>>>>>>> I then hit the https://loadbalancer.domain.com/static from my
>>>>>>>>>>> Win10
>>>>>>>>>>> laptop
>>>>>>>>>>> (10.1.1.2 example)
>>>>>>>>>>> and I see this in the HTTP server #1 log, but NOTHING in the
>>>>>>>>>>> tomcat
>>>>>>>>>>> access
>>>>>>>>>>> log.
>>>>>>>>>>> 10.1.1.2 - - [22/Feb/2017:10:20:02 -0800] "-" 408 -
>>>>>>>>>>>
>>>>>>>>>>> So 408. Timeout. Hmm... Why? We KNOW that it can connect from
>>>>>>>>>>> http ->
>>>>>>>>>>> tomcat:18080 perfectly. So is it a timeout BACK to the F5?
>>>>>>>>>>> That i
>>>>>>>>>>> dont
>>>>>>>>>>> know yet.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> It looks as if, in that case, httpd is trying to connect to
>>>>>>>>>> tomcat,
>>>>>>>>>> but
>>>>>>>>>> - either it /is/ connecting, and sending the request, but tomcat
>>>>>>>>>> never answers
>>>>>>>>>> - or it is connecting to something else, which isn't tomcat
>>>>>>>>>> (but which accepts the request and doesn't answer either)
>>>>>>>>>>
>>>>>>>>>> I am not *sure*, but I believe, that if httpd was trying to
>>>>>>>>>> connect
>>>>>>>>>> to something that isn't tomcat, and is not listening on port
>>>>>>>>>> 18080, it
>>>>>>>>>> would get a different error, and a different logfile message. So
>>>>>>>>>> it would
>>>>>>>>>> seem that at least the TCP connection works.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Is the F5 loadbalancer configured to make reverse proxy to a SSL
>>>>>>>>> http?
>>>>>>>>> The directive SSLProxyEngine onthe F5 loadbalencer has to be
>>>>>>>>> enabled.
>>>>>>>>> http://httpd.apache.org/docs/current/mod/mod_ssl.html#sslpro
>>>>>>>>> xyengine
>>>>>>>>>
>>>>>>>>> If the loadbalancer it's delegating the SSL work to the server1
>>>>>>>>> server1 https listen = 172.1.1.1:23270
>>>>>>>>>
>>>>>>>>> then the F5 could be connecting as http regular TCP connection to a
>>>>>>>>> SSL TCP one and therefor the SSL handshake it's not happening.
>>>>>>>>>
>>>>>>>>> Antonio
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Now there is still a puzzle there : when you connect as
>>>>>>>>>> browser --> httpd --> tomcat
>>>>>>>>>> then everything works : httpd connects to tomcat, sends a request,
>>>>>>>>>> gets a response, and returns it to the browser.
>>>>>>>>>>
>>>>>>>>>> But when you do the same as :
>>>>>>>>>> browser --> F5 --> httpd --> tomcat
>>>>>>>>>> it "does not work", and the problem seems to be at httpd --> tomat
>>>>>>>>>> Why would it make a difference in httpd --> tomcat, if it is
>>>>>>>>>> either
>>>>>>>>>> the browser or the F5 which sends the request to httpd ?
>>>>>>>>>> the httpd --> tomcat connection should be the same, no matter
>>>>>>>>>> what.
>>>>>>>>>>
>>>>>>>>>> Hmm, for now as puzzled as you are.
>>>>>>>>>> Maybe we have a quantum phenomenon here, where the mere fact of
>>>>>>>>>> you
>>>>>>>>>> looking at the logfile, changes what happened beforehand. Just
>>>>>>>>>> kidding.
>>>>>>>>>>
>>>>>>>>>> Informational note : the Apache access log line, is written when
>>>>>>>>>> Apache *has finished* processing the request (including the
>>>>>>>>>> possible
>>>>>>>>>> round-trip through tomcat) and has sent the response to the
>>>>>>>>>> client. (That's
>>>>>>>>>> because, for example, it can also write the time spent, and the
>>>>>>>>>> size of the
>>>>>>>>>> response; so it has to know those before it can write the log
>>>>>>>>>> line).
>>>>>>>>>> I presume that it is the same for tomcat.
>>>>>>>>>> Just mentioning this, so that you would maybe have a second look
>>>>>>>>>> at
>>>>>>>>>> the tomcat access log, after a few minutes, to make sure that
>>>>>>>>>> there really
>>>>>>>>>> is no corresponding line there.
>>>>>>>>>> It could be that tomcat actually /is/ receiving the request, and
>>>>>>>>>> processing it, but that it takes a long time, and that httpd
>>>>>>>>>> tires of
>>>>>>>>>> waiting and already times out in the meantime. In such a case,
>>>>>>>>>> you would
>>>>>>>>>> probably see an error some time later in the tomcat error log,
>>>>>>>>>> when it is
>>>>>>>>>> trying to eventually return the response, and finds a closed
>>>>>>>>>> connection
>>>>>>>>>> instead.
>>>>>>>>>> Now again, why it would do that in one case and not the other, is
>>>>>>>>>> still a mystery.
>>>>>>>>>>
>>>>>>>>>> Another note : in the "timeout" log line above, httpd is not
>>>>>>>>>> showing
>>>>>>>>>> the request URL which triggered this timeout.
>>>>>>>>>> a) it may be showing some additional information in the Apache
>>>>>>>>>> error
>>>>>>>>>> log
>>>>>>>>>> b) these exists an httpd add-on module (mod_log_forensic ?) which
>>>>>>>>>> could provide more details in the log, about what happens
>>>>>>>>>> precisely
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Tue, Feb 21, 2017 at 3:39 PM, André Warnier (tomcat) <
>>>>>>>>>>> aw@ice-sa.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> On 21.02.2017 23:28, Aaron Gray wrote:
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Antonio: The Tomcat server has no knowledge of the F5, or that
>>>>>>>>>>>> it
>>>>>>>>>>>>
>>>>>>>>>>>>> is
>>>>>>>>>>>>> being
>>>>>>>>>>>>> fronted by an Apache HTTP Server. I do SSL termination in
>>>>>>>>>>>>> Apache
>>>>>>>>>>>>> HTTP
>>>>>>>>>>>>> Server, and clear-text from HTTP to Tomcat.
>>>>>>>>>>>>> My redirect port for the normal HTTP listen in Tomcat is
>>>>>>>>>>>>> commented
>>>>>>>>>>>>> out.
>>>>>>>>>>>>> <Connector port="18080" protocol="HTTP/1.1"
>>>>>>>>>>>>> connectionTimeout="20000" />
>>>>>>>>>>>>> <!-- A "Connector" using the shared thread pool-->
>>>>>>>>>>>>> <!--
>>>>>>>>>>>>> <Connector executor="tomcatThreadPool"
>>>>>>>>>>>>> port="8080" protocol="HTTP/1.1"
>>>>>>>>>>>>> connectionTimeout="20000"
>>>>>>>>>>>>> redirectPort="8443" />
>>>>>>>>>>>>> -->
>>>>>>>>>>>>>
>>>>>>>>>>>>> Andre:
>>>>>>>>>>>>> The URL I am using is https://loadbalancer.domain.com
>>>>>>>>>>>>> It is listening on port 80 and 443, if you hit 80, internally
>>>>>>>>>>>>> it
>>>>>>>>>>>>> redirects
>>>>>>>>>>>>> you to 443. No SSL cert on the F5 load balancer. It simply
>>>>>>>>>>>>> sends
>>>>>>>>>>>>> the
>>>>>>>>>>>>> traffic to one of the two HTTP servers (round-robin, also tried
>>>>>>>>>>>>> persistence, no difference). The HTTP server is listening only
>>>>>>>>>>>>> for HTTPS
>>>>>>>>>>>>> on 23270/tcp.
>>>>>>>>>>>>> When I access these DMZ webservers via the F5 load balancer (to
>>>>>>>>>>>>> which I
>>>>>>>>>>>>> dont have access to, but the network folks configure for me),
>>>>>>>>>>>>> it
>>>>>>>>>>>>> hangs.
>>>>>>>>>>>>> Eventually returns: server1 https listen = 172.1.1.1:23270
>>>>>>>>>>>>>
>>>>>>>>>>>>> https://loadbalancer.domain.com:23270/SelfService
>>>>>>>>>>>>> cant load.
>>>>>>>>>>>>>
>>>>>>>>>>>>> No idea why the URL is being re-written with the ":23270".
>>>>>>>>>>>>> I added static content to the server.xml on 10.1.1.1 (Tomcat)
>>>>>>>>>>>>> to
>>>>>>>>>>>>> test:
>>>>>>>>>>>>> <Context docBase="/path/to/tomcat/static" path="/static" />
>>>>>>>>>>>>> Then put a simple index.html in there. Accessing via the
>>>>>>>>>>>>> Apache
>>>>>>>>>>>>> Web
>>>>>>>>>>>>> Servers works fine, but if you hit it with the Load Balancer it
>>>>>>>>>>>>> once
>>>>>>>>>>>>> again
>>>>>>>>>>>>> adds the https://loadbalancer.domain.com:23270/static
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hitting https://loadbalancer.domain.com
>>>>>>>>>>>>> I see my "Hello world!" which is all that is in index.html.
>>>>>>>>>>>>> This
>>>>>>>>>>>>> is the
>>>>>>>>>>>>> DocumentRoot of HTTP, and *not* proxied over at this time.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> So in this case, there is no delay, and you get the Apache
>>>>>>>>>>>> httpd-hosted
>>>>>>>>>>>> "index.html" containing "Hello World. Right ?
>>>>>>>>>>>>
>>>>>>>>>>>> Only
>>>>>>>>>>>>
>>>>>>>>>>>> /SelfService and /static are proxied
>>>>>>>>>>>>
>>>>>>>>>>>>> /static just being my test of static content, but still served
>>>>>>>>>>>>> up
>>>>>>>>>>>>> by
>>>>>>>>>>>>> Tomcat..
>>>>>>>>>>>>>
>>>>>>>>>>>>> It's exactly 30 seconds before the page cannot be loaded when
>>>>>>>>>>>>> trying
>>>>>>>>>>>>> anything proxied to Tomcat, but also accessed via the F5 load
>>>>>>>>>>>>> balancer.
>>>>>>>>>>>>> Not sure where the 30 seconds comes from; perhaps a load
>>>>>>>>>>>>> balancer
>>>>>>>>>>>>> time
>>>>>>>>>>>>> out,
>>>>>>>>>>>>> as I dont see a "30" in my httpd configurations or my tomcat
>>>>>>>>>>>>> server.xml
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> You can certainly look at the Apache httpd logs, and the tomcat
>>>>>>>>>>>>>
>>>>>>>>>>>> logs, to
>>>>>>>>>>>> see if you get a request or not.
>>>>>>>>>>>> In Apache httpd, you can set the loglevel individually for
>>>>>>>>>>>> mod_proxy (if
>>>>>>>>>>>> you are running v 2.4), and it should show something if it gets
>>>>>>>>>>>> this
>>>>>>>>>>>> request and forwards it to tomcat.
>>>>>>>>>>>> In tomcat, you can either enable an access log (which will show
>>>>>>>>>>>> if
>>>>>>>>>>>> it
>>>>>>>>>>>> receives this request), or you could temporarily remove/rename
>>>>>>>>>>>> the
>>>>>>>>>>>> /static
>>>>>>>>>>>> webapp. This way, it should trigger an error "not found" which
>>>>>>>>>>>> you
>>>>>>>>>>>> would
>>>>>>>>>>>> also see in the error log.
>>>>>>>>>>>>
>>>>>>>>>>>> There should be nothing between them to hinder it. We have many
>>>>>>>>>>>> load
>>>>>>>>>>>>
>>>>>>>>>>>> balancers and this one specifically you dont need to open any
>>>>>>>>>>>>> firewall
>>>>>>>>>>>>> requests for the specific networks the HTTP servers are on. I
>>>>>>>>>>>>> did
>>>>>>>>>>>>> have to
>>>>>>>>>>>>> get the firewall opened up to allow me to hit
>>>>>>>>>>>>> https://loadbalancer.domain.com because the VIP for "
>>>>>>>>>>>>> loadbalancer.domain.com"
>>>>>>>>>>>>> is in the DMZ, and my Desktop & VPN networks cannot hit it on
>>>>>>>>>>>>> 80/443
>>>>>>>>>>>>> without opening holes. But beyond that, any connection from
>>>>>>>>>>>>> the
>>>>>>>>>>>>> F5 to the
>>>>>>>>>>>>> HTTP Server should be 100% open bi-directional, since same
>>>>>>>>>>>>> subnet..
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> But something isn't working, otherwise you would not be asking.
>>>>>>>>>>>>>
>>>>>>>>>>>> So,
>>>>>>>>>>>>
>>>>>>>>>>>> a) hitting the tomcat webapps through httpd seems to be working
>>>>>>>>>>>> fine
>>>>>>>>>>>> (browser -> httpd:23270 -> tomcat:18080 -> webapp or static)
>>>>>>>>>>>>
>>>>>>>>>>>> b) hitting a non-proxied-to-tomcat resource of httpd seems to
>>>>>>>>>>>> work
>>>>>>>>>>>> fine
>>>>>>>>>>>> too, even through the F5
>>>>>>>>>>>> (browser -> F5:443 -> httpd:23270 -> html page)
>>>>>>>>>>>>
>>>>>>>>>>>> c) it is only when you do :
>>>>>>>>>>>> (browser -> F5:443 -> httpd:23270 -> tomcat:18080 -> webapp
>>>>>>>>>>>> or
>>>>>>>>>>>> static)
>>>>>>>>>>>> that you see this issue
>>>>>>>>>>>>
>>>>>>>>>>>> It would really help if you looked in the logs of both httpd and
>>>>>>>>>>>> tomcat,
>>>>>>>>>>>> and checked for differences betweens cases a, b and c above.
>>>>>>>>>>>>
>>>>>>>>>>>> I believe that the F5 message with the port 23270 is a minor
>>>>>>>>>>>> issue,
>>>>>>>>>>>> of
>>>>>>>>>>>> information disclosure by the F5, that it should not disclose.
>>>>>>>>>>>>
>>>>>>>>>>>> But the reason why it returns this error is obviously that in
>>>>>>>>>>>> that
>>>>>>>>>>>> case,
>>>>>>>>>>>> it does not get a response from his request to httpd.
>>>>>>>>>>>> The reason for this response not coming back to the F5 (in case
>>>>>>>>>>>> c
>>>>>>>>>>>> only),
>>>>>>>>>>>> can be due to either httpd or tomcat. But F5 doesn't know about
>>>>>>>>>>>> tomcat. So
>>>>>>>>>>>> for the F5, it is httpd which is not responding. Thus,
>>>>>>>>>>>> - either httpd is never getting the request from the F5
>>>>>>>>>>>> (unlikely,
>>>>>>>>>>>> because
>>>>>>>>>>>> in b above it gets it and responds)
>>>>>>>>>>>> - or httpd is getting the request from the F5, but not
>>>>>>>>>>>> forwarding
>>>>>>>>>>>> it to
>>>>>>>>>>>> tomcat, but also not returning an immediate error response to F5
>>>>>>>>>>>> (which
>>>>>>>>>>>> seems also unlikely, because of a and b)
>>>>>>>>>>>> - or httpd is getting the request, forwarding it to tomcat, but
>>>>>>>>>>>> not
>>>>>>>>>>>> getting a response from tomcat. So
>>>>>>>>>>>> - either tomcat is never getting the request from httpd
>>>>>>>>>>>> (but in
>>>>>>>>>>>> a, it
>>>>>>>>>>>> gets it)
>>>>>>>>>>>> - or tomcat is getting the request from httpd, but not
>>>>>>>>>>>> responding (but
>>>>>>>>>>>> in a, it does)
>>>>>>>>>>>> - or tomcat is getting the request and responding, but the
>>>>>>>>>>>> response
>>>>>>>>>>>> never gets back to httpd (but in a, it does)
>>>>>>>>>>>>
>>>>>>>>>>>> So if a and b and c are all accurate, there is something
>>>>>>>>>>>> apparently
>>>>>>>>>>>> illogical happening.
>>>>>>>>>>>> This would lead to the conclusion that a and b and c cannot all
>>>>>>>>>>>> be
>>>>>>>>>>>> accurate.
>>>>>>>>>>>>
>>>>>>>>>>>> The logs.. ?
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, Feb 21, 2017 at 2:05 PM, Antonio S. Cofino <
>>>>>>>>>>>> cofinoa@gmail.com>
>>>>>>>>>>>>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Aaron, on tomcat instances change the redirectPort attributte
>>>>>>>>>>>>> on
>>>>>>>>>>>>> the http
>>>>>>>>>>>>>
>>>>>>>>>>>>> conectó to the loabbalancer's port 443
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> My guess is that your webapp has restriction rule requesting
>>>>>>>>>>>>>> SSL
>>>>>>>>>>>>>> con
>>>>>>>>>>>>>> fidntial channel. Therefore the non-confidential to the 18080
>>>>>>>>>>>>>> port from
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>> balancer are redirected to the 23270 port, but it should be
>>>>>>>>>>>>>> 443.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Antonio
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> El 21/2/2017 19:46, "Aaron Gray" <aa...@gmail.com>
>>>>>>>>>>>>>> escribió:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I have an application server from a vendor that comes bundled
>>>>>>>>>>>>>> with an
>>>>>>>>>>>>>> additional Apache Tomcat server. The webapp SelfService.war
>>>>>>>>>>>>>> is
>>>>>>>>>>>>>> vendor
>>>>>>>>>>>>>> supplied too.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Here's my problem (IP's replaced to protect the innocent):
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> networks:
>>>>>>>>>>>>>> DMZ=172.x.x.x
>>>>>>>>>>>>>> INTERNAL=10.x.x.x
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> server1 https listen = 172.1.1.1:23270
>>>>>>>>>>>>>> server2 https listen = 172.1.1.2:23270
>>>>>>>>>>>>>> F5 load balancer hostname = loadbalancer.domain.com:443
>>>>>>>>>>>>>> backend tomcat server = 10.1.1.1:18080
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> mod_proxy configuration:
>>>>>>>>>>>>>> ProxyPass /SelfService http://10.1.1.1:18080/SelfService
>>>>>>>>>>>>>> ProxyPassReverse /SelfService http://10.1.1.1:18080/SelfServ
>>>>>>>>>>>>>> ice
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> When I access these DMZ webservers which mod_proxy back to
>>>>>>>>>>>>>> Apache
>>>>>>>>>>>>>> Tomcat
>>>>>>>>>>>>>> as:
>>>>>>>>>>>>>> https://172.1.1.1:23270/SelfService
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>> https://172.1.1.2:23270/SelfService <
>>>>>>>>>>>>>> https://172.1.1.1:23270/SelfService
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> They load properly. Perfectly, every time!
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> When I access these DMZ webservers via the F5 load balancer
>>>>>>>>>>>>>> (to
>>>>>>>>>>>>>> which I
>>>>>>>>>>>>>> dont have access to, but the network folks configure for me),
>>>>>>>>>>>>>> it
>>>>>>>>>>>>>> hangs.
>>>>>>>>>>>>>> Eventually returns:
>>>>>>>>>>>>>> https://loadbalancer.domain.com:23270/SelfService
>>>>>>>>>>>>>> cant load.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> No idea why the URL is being re-written with the ":23270".
>>>>>>>>>>>>>> I added static content to the server.xml on 10.1.1.1 (Tomcat)
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>> test:
>>>>>>>>>>>>>> <Context docBase="/path/to/tomcat/static" path="/static" />
>>>>>>>>>>>>>> Then put a simple index.html in there. Accessing via the
>>>>>>>>>>>>>> Apache
>>>>>>>>>>>>>> Web
>>>>>>>>>>>>>> Servers works fine, but if you hit it with the Load Balancer
>>>>>>>>>>>>>> it
>>>>>>>>>>>>>> once
>>>>>>>>>>>>>> again
>>>>>>>>>>>>>> adds the https://loadbalancer.domain.com:23270/static
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Do you have any thoughts? Thanks so much, I have been working
>>>>>>>>>>>>>> with this
>>>>>>>>>>>>>> for weeks now with no success
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>> ------------------------------------------------------------
>>>>>>>>>>>> ---------
>>>>>>>>>>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>>>>>>>>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> ------------------------------------------------------------
>>>>>>>>>> ---------
>>>>>>>>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>>>>>>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> ------------------------------------------------------------
>>>>>>>>> ---------
>>>>>>>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>>>>>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>>
>
Re: Strange URL rewrite when reverse proxy with Apache HTTP Server
Posted by Aaron Gray <aa...@gmail.com>.
Andre, that is all very educational and I feel even when this is resolved,
that I have learned a good amount here; so I thank you for everything.
I have an update. I decided to shutdown the vendor included/supplied
7.0.50 release of Tomcat. I extracted vanilla 7.0.75 tomcat, updated
server.xml to change port from 8080 to 18080, and added the static line
(created a new /static location like:
/app/home/agray/apache-tomcat-7.0.75/static
Started that tomcat
https://loadbalancer.domain.com/static
loads PERFECTLY! Whaaaatttt? Now I guess I could be investigating the
differences between these two environments. I do remember though they had
a document that I followed a long time ago when the release of Tomcat they
included had a major vulnerability in it (years ago) to copy their specific
.war and changes in to a vanilla Tomcat and deploy it there. I think I
want to do that. I have escalated to their developers, since I can prove a
vanilla tomcat can serve up my static content through the browser -> F5 ->
apache http proxy server -> app server yet their included one cant
without 302 and adding the :port and having to remove it, etc.
Wild. I'll update soon.
BTW, hope you got some sleep after last night, but want to thank you again.
On Thu, Feb 23, 2017 at 4:00 PM, André Warnier (tomcat) <aw...@ice-sa.com>
wrote:
> To avoid getting totally confused, you may want to read this explanation
> first :
> https://wiki.apache.org/httpd/DirectoryListings
>
> In other words, when you send this request from the browser :
> https://loadbalancer.domain.com/SelfService
>
> The first response that /should/ come back, would be a "re-direct"
> response from the (final) server, with a special HTTP header added :
> Location: https://loadbalancer.domain.com/SelfService/
>
> Then your browser should re-issue the request automatically, this time to :
> https://loadbalancer.domain.com/SelfService/
> (with the trailing "/" added)
>
> Then, in a second step, what should happen is :
>
> Your browser indicates a URL, which on the server resolves to a directory.
> The server /may/ be configured in such a way, that for this kind of
> request, it returns an auto-generated list of all the files in that
> directory, automatically.
> Or it may be configured to return instead, a specific html page.
> (That what you do in Apache, with the DirectoryIndex directive)
> And if the server is not configured at all for that, it would probably
> return a response "Forbidden", for security reasons (you probably don't
> want someone to be able to list the files in that directory, unless you
> specifically configured the server to allow that).
>
> Or, the server may be configured so that for such a request, it defaults
> to running some standard servlet which is part of the application, and
> which would return the application starting page.
> That would probably be the case in your "/SelfService" tomcat webapp.
>
> So, the 302 responses which you get in some cases, are probably normal, as
> explained by all the above.
> What is definitely /not/ normal, is that port that you get either in these
> redirect responses, or in the error messages. Because that means that,
> somewhere in the chain, one of the elements is not doing its job properly
> when proxying.
> The point of the ProxyPassReverse directive, is precisely that : ensure
> that if the "proxied to" server returns a re-direct response, the
> "Location:" header that is in that redirect response would be properly
> edited before returning that redirect response back up the chain, in such a
> way that the ultimate client would not try next, to access a URL that leads
> nowhere.
>
> And now it's really late..
>
>
>
> On 24.02.2017 00:06, André Warnier (tomcat) wrote:
>
>> On 23.02.2017 21:17, Aaron Gray wrote:
>>
>>> Another weird thing...
>>>
>>> If you go to:
>>> https://loadbalancer.domain.com/SelfService
>>> If I am viewing the headers immedaitely I see 302 not found.
>>>
>>
>> 302 is not "not found". It is "Found", and it is a Redirect response.
>> See : https://en.wikipedia.org/wiki/HTTP_302
>> for a good explanation.
>> (But stop after the second paragraph, otherwise you'll get confused
>> again..)
>>
>> Then I wait
>>
>>> the 30 sec for it to totally fail and change the URL in the browser to
>>> https://loadbalancer.domain.com:232700/SelfService
>>>
>>> If you just remove the :232700 and hit Enter, it then loads properly.
>>> See
>>> immedaitly 200 OK and it is fine. Don't navigate anywhere else like to
>>> https://loadbalancer.domain.com/static or else you'll have to repeat the
>>> same thing as above with the :232700 and removing it, hit Enter.
>>>
>>> Weird.
>>>
>>> Of course none of this is an issue if you dont hit it with the F5 Load
>>> Balancer to start.
>>>
>>
>> Yes. There seems to be something weird *when a Redirect response is being
>> sent back by
>> tomcat*, and has to go back through the load balancer.
>> And it may be linked to the fact that the load-balancer does not seem to
>> remove this
>> :23270 port which eventually reaches the browser.
>>
>>
>>
>>
>>> On Thu, Feb 23, 2017 at 10:24 AM, Aaron Gray <aa...@gmail.com>
>>> wrote:
>>>
>>> I randomly tried https://loadbalancer.domain.com/SelfService/index.html
>>>>
>>>> It works perfectly. No issues. I have no idea why its needed to add on
>>>> the /index.html (i'm lucky this exists in the tomcat .war, as it was a
>>>> guess). This is visually not appealing, but workable. Maybe I can do
>>>> some
>>>> url rewrite or something, IF this cant be fixed normally?
>>>>
>>>> On Thu, Feb 23, 2017 at 10:10 AM, Aaron Gray <aa...@gmail.com>
>>>> wrote:
>>>>
>>>> Okay, guys, please accept my most sincere apology on this thing. I just
>>>>> figured 1 thing out.
>>>>>
>>>>> browser -> f5:443 -> httpd (23270, https) -> tomcat (http)
>>>>> using the /static works! (see below)
>>>>>
>>>>> If /static/index.html wasnt specified, then it hangs, then comes back
>>>>> as
>>>>> https://loadbalancer.domain.com:23270/static
>>>>> But if you go https://loadbalancer.domain.com/static/index.html it
>>>>> works!!!
>>>>>
>>>>> DirectoryIndex index.html
>>>>> is present. because if you just go to https://webserver1:23270/static
>>>>> it works fine, no need for the extra /index.html
>>>>>
>>>>> not sure why that is different there.
>>>>>
>>>>> I only found this out using curl and just happened to think of it.
>>>>> So you see the difference, from the tomcat access log, in the two
>>>>> variations.
>>>>>
>>>>> 172.18.160.66 - - [23/Feb/2017:10:00:48 -0800] "GET /static HTTP/1.1"
>>>>> 302
>>>>> -
>>>>> 172.18.160.66 - - [23/Feb/2017:10:03:04 -0800] "GET /static/index.html
>>>>> HTTP/1.1" 200 32
>>>>>
>>>>> Still /SelfService wont work though
>>>>> I cant think of a way to do /SelfService/something...
>>>>> Since it is a web app and i am no developer and its all from the vendor
>>>>> (BMC Software) and it never shows naything other than /SelfService in
>>>>> the
>>>>> URL. I dont think its a frame, but once again i dont know these
>>>>> things. I
>>>>> know they told me it is HTML5, as it was using Silverlight back in the
>>>>> older release.
>>>>>
>>>>>
>>>>> On Thu, Feb 23, 2017 at 9:22 AM, Aaron Gray <aa...@gmail.com>
>>>>> wrote:
>>>>>
>>>>> Working with my F5 guy, we had an idea, since 80/tcp and 443/tcp were
>>>>>> already open to the VIP on the F5, we simply turned of 80 -> 443
>>>>>> redirect
>>>>>> on the F5, and then configured the F5 to use the non-HTTP port in
>>>>>> Apache
>>>>>> HTTP Server (the two backend servers). So its HTTP the entire way
>>>>>> through,
>>>>>> and the same issue persisted.
>>>>>>
>>>>>> browser -> F5(http) -> httpd(http port) -> tomcat(http port, nonssl)
>>>>>>
>>>>>> So i think i can rule out any possibility that the fact I was
>>>>>> terminating SSL in Apache HTTP Server then proxying clear-text back
>>>>>> as a
>>>>>> reason. It worked fine like I said if you just go direct to the HTTP
>>>>>> Server and slip the F5. And F5 works landing the /healthcheck.html
>>>>>> (HTTP
>>>>>> KeepAlive in the root, which isnt proxied), so all good there. Just
>>>>>> cant
>>>>>> /SelfService (configured w/ Proxy)
>>>>>>
>>>>>> thanks for your support here though.
>>>>>>
>>>>>> On Thu, Feb 23, 2017 at 9:12 AM, Aaron Gray <aa...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>> SSLProxyEngine On
>>>>>>> Was already turned on this entire time inside the ssl.conf (I include
>>>>>>> it) VirtualHost section.
>>>>>>>
>>>>>>> I am debating turning on HTTPS in Tomcat on the backend 10.x.x.x app
>>>>>>> server, and then HTTPS the whole way through and see if that makes
>>>>>>> any
>>>>>>> difference. I may need to request a new firewall to be opened,
>>>>>>> which is
>>>>>>> not able to open until 3/2. Gonna see.
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Feb 23, 2017 at 8:22 AM, Antonio S. Cofiño <
>>>>>>> cofinoa@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>
>>>>>>>> On 23/02/17 12:43, André Warnier (tomcat) wrote:
>>>>>>>>
>>>>>>>> On 22.02.2017 19:22, Aaron Gray wrote:
>>>>>>>>>
>>>>>>>>> So this is interesting.
>>>>>>>>>>
>>>>>>>>>> So from HTTP server #1 (172.1.1.1 example) I hit:
>>>>>>>>>> https://172.1.1.1:23270/static and I see this in the HTTP log:
>>>>>>>>>> 172.1.1.1 - - [22/Feb/2017:10:14:48 -0800] "GET /static/ HTTP/1.1"
>>>>>>>>>> 200 32
>>>>>>>>>> I see this in the Tomcat log:
>>>>>>>>>> 172.1.1.1 - - [22/Feb/2017:10:14:48 -0800] "GET /static/ HTTP/1.1"
>>>>>>>>>> 200 32
>>>>>>>>>> ((( Yes I know they look identical, but the logformat is the same,
>>>>>>>>>> so it
>>>>>>>>>> comes out looking the same, but they are two diff log files
>>>>>>>>>> totally
>>>>>>>>>> )))
>>>>>>>>>>
>>>>>>>>>> I then hit the https://loadbalancer.domain.com/static from my
>>>>>>>>>> Win10
>>>>>>>>>> laptop
>>>>>>>>>> (10.1.1.2 example)
>>>>>>>>>> and I see this in the HTTP server #1 log, but NOTHING in the
>>>>>>>>>> tomcat
>>>>>>>>>> access
>>>>>>>>>> log.
>>>>>>>>>> 10.1.1.2 - - [22/Feb/2017:10:20:02 -0800] "-" 408 -
>>>>>>>>>>
>>>>>>>>>> So 408. Timeout. Hmm... Why? We KNOW that it can connect from
>>>>>>>>>> http ->
>>>>>>>>>> tomcat:18080 perfectly. So is it a timeout BACK to the F5? That
>>>>>>>>>> i
>>>>>>>>>> dont
>>>>>>>>>> know yet.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> It looks as if, in that case, httpd is trying to connect to tomcat,
>>>>>>>>> but
>>>>>>>>> - either it /is/ connecting, and sending the request, but tomcat
>>>>>>>>> never answers
>>>>>>>>> - or it is connecting to something else, which isn't tomcat
>>>>>>>>> (but which accepts the request and doesn't answer either)
>>>>>>>>>
>>>>>>>>> I am not *sure*, but I believe, that if httpd was trying to connect
>>>>>>>>> to something that isn't tomcat, and is not listening on port
>>>>>>>>> 18080, it
>>>>>>>>> would get a different error, and a different logfile message. So
>>>>>>>>> it would
>>>>>>>>> seem that at least the TCP connection works.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> Is the F5 loadbalancer configured to make reverse proxy to a SSL
>>>>>>>> http?
>>>>>>>> The directive SSLProxyEngine onthe F5 loadbalencer has to be
>>>>>>>> enabled.
>>>>>>>> http://httpd.apache.org/docs/current/mod/mod_ssl.html#sslpro
>>>>>>>> xyengine
>>>>>>>>
>>>>>>>> If the loadbalancer it's delegating the SSL work to the server1
>>>>>>>> server1 https listen = 172.1.1.1:23270
>>>>>>>>
>>>>>>>> then the F5 could be connecting as http regular TCP connection to a
>>>>>>>> SSL TCP one and therefor the SSL handshake it's not happening.
>>>>>>>>
>>>>>>>> Antonio
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Now there is still a puzzle there : when you connect as
>>>>>>>>> browser --> httpd --> tomcat
>>>>>>>>> then everything works : httpd connects to tomcat, sends a request,
>>>>>>>>> gets a response, and returns it to the browser.
>>>>>>>>>
>>>>>>>>> But when you do the same as :
>>>>>>>>> browser --> F5 --> httpd --> tomcat
>>>>>>>>> it "does not work", and the problem seems to be at httpd --> tomat
>>>>>>>>> Why would it make a difference in httpd --> tomcat, if it is either
>>>>>>>>> the browser or the F5 which sends the request to httpd ?
>>>>>>>>> the httpd --> tomcat connection should be the same, no matter what.
>>>>>>>>>
>>>>>>>>> Hmm, for now as puzzled as you are.
>>>>>>>>> Maybe we have a quantum phenomenon here, where the mere fact of you
>>>>>>>>> looking at the logfile, changes what happened beforehand. Just
>>>>>>>>> kidding.
>>>>>>>>>
>>>>>>>>> Informational note : the Apache access log line, is written when
>>>>>>>>> Apache *has finished* processing the request (including the
>>>>>>>>> possible
>>>>>>>>> round-trip through tomcat) and has sent the response to the
>>>>>>>>> client. (That's
>>>>>>>>> because, for example, it can also write the time spent, and the
>>>>>>>>> size of the
>>>>>>>>> response; so it has to know those before it can write the log
>>>>>>>>> line).
>>>>>>>>> I presume that it is the same for tomcat.
>>>>>>>>> Just mentioning this, so that you would maybe have a second look at
>>>>>>>>> the tomcat access log, after a few minutes, to make sure that
>>>>>>>>> there really
>>>>>>>>> is no corresponding line there.
>>>>>>>>> It could be that tomcat actually /is/ receiving the request, and
>>>>>>>>> processing it, but that it takes a long time, and that httpd tires
>>>>>>>>> of
>>>>>>>>> waiting and already times out in the meantime. In such a case, you
>>>>>>>>> would
>>>>>>>>> probably see an error some time later in the tomcat error log,
>>>>>>>>> when it is
>>>>>>>>> trying to eventually return the response, and finds a closed
>>>>>>>>> connection
>>>>>>>>> instead.
>>>>>>>>> Now again, why it would do that in one case and not the other, is
>>>>>>>>> still a mystery.
>>>>>>>>>
>>>>>>>>> Another note : in the "timeout" log line above, httpd is not
>>>>>>>>> showing
>>>>>>>>> the request URL which triggered this timeout.
>>>>>>>>> a) it may be showing some additional information in the Apache
>>>>>>>>> error
>>>>>>>>> log
>>>>>>>>> b) these exists an httpd add-on module (mod_log_forensic ?) which
>>>>>>>>> could provide more details in the log, about what happens precisely
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, Feb 21, 2017 at 3:39 PM, André Warnier (tomcat) <
>>>>>>>>>> aw@ice-sa.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> On 21.02.2017 23:28, Aaron Gray wrote:
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Antonio: The Tomcat server has no knowledge of the F5, or that
>>>>>>>>>>> it
>>>>>>>>>>>
>>>>>>>>>>>> is
>>>>>>>>>>>> being
>>>>>>>>>>>> fronted by an Apache HTTP Server. I do SSL termination in
>>>>>>>>>>>> Apache
>>>>>>>>>>>> HTTP
>>>>>>>>>>>> Server, and clear-text from HTTP to Tomcat.
>>>>>>>>>>>> My redirect port for the normal HTTP listen in Tomcat is
>>>>>>>>>>>> commented
>>>>>>>>>>>> out.
>>>>>>>>>>>> <Connector port="18080" protocol="HTTP/1.1"
>>>>>>>>>>>> connectionTimeout="20000" />
>>>>>>>>>>>> <!-- A "Connector" using the shared thread pool-->
>>>>>>>>>>>> <!--
>>>>>>>>>>>> <Connector executor="tomcatThreadPool"
>>>>>>>>>>>> port="8080" protocol="HTTP/1.1"
>>>>>>>>>>>> connectionTimeout="20000"
>>>>>>>>>>>> redirectPort="8443" />
>>>>>>>>>>>> -->
>>>>>>>>>>>>
>>>>>>>>>>>> Andre:
>>>>>>>>>>>> The URL I am using is https://loadbalancer.domain.com
>>>>>>>>>>>> It is listening on port 80 and 443, if you hit 80, internally it
>>>>>>>>>>>> redirects
>>>>>>>>>>>> you to 443. No SSL cert on the F5 load balancer. It simply
>>>>>>>>>>>> sends
>>>>>>>>>>>> the
>>>>>>>>>>>> traffic to one of the two HTTP servers (round-robin, also tried
>>>>>>>>>>>> persistence, no difference). The HTTP server is listening only
>>>>>>>>>>>> for HTTPS
>>>>>>>>>>>> on 23270/tcp.
>>>>>>>>>>>> When I access these DMZ webservers via the F5 load balancer (to
>>>>>>>>>>>> which I
>>>>>>>>>>>> dont have access to, but the network folks configure for me), it
>>>>>>>>>>>> hangs.
>>>>>>>>>>>> Eventually returns: server1 https listen = 172.1.1.1:23270
>>>>>>>>>>>>
>>>>>>>>>>>> https://loadbalancer.domain.com:23270/SelfService
>>>>>>>>>>>> cant load.
>>>>>>>>>>>>
>>>>>>>>>>>> No idea why the URL is being re-written with the ":23270".
>>>>>>>>>>>> I added static content to the server.xml on 10.1.1.1 (Tomcat) to
>>>>>>>>>>>> test:
>>>>>>>>>>>> <Context docBase="/path/to/tomcat/static" path="/static" />
>>>>>>>>>>>> Then put a simple index.html in there. Accessing via the Apache
>>>>>>>>>>>> Web
>>>>>>>>>>>> Servers works fine, but if you hit it with the Load Balancer it
>>>>>>>>>>>> once
>>>>>>>>>>>> again
>>>>>>>>>>>> adds the https://loadbalancer.domain.com:23270/static
>>>>>>>>>>>>
>>>>>>>>>>>> Hitting https://loadbalancer.domain.com
>>>>>>>>>>>> I see my "Hello world!" which is all that is in index.html. This
>>>>>>>>>>>> is the
>>>>>>>>>>>> DocumentRoot of HTTP, and *not* proxied over at this time.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> So in this case, there is no delay, and you get the Apache
>>>>>>>>>>> httpd-hosted
>>>>>>>>>>> "index.html" containing "Hello World. Right ?
>>>>>>>>>>>
>>>>>>>>>>> Only
>>>>>>>>>>>
>>>>>>>>>>> /SelfService and /static are proxied
>>>>>>>>>>>
>>>>>>>>>>>> /static just being my test of static content, but still served
>>>>>>>>>>>> up
>>>>>>>>>>>> by
>>>>>>>>>>>> Tomcat..
>>>>>>>>>>>>
>>>>>>>>>>>> It's exactly 30 seconds before the page cannot be loaded when
>>>>>>>>>>>> trying
>>>>>>>>>>>> anything proxied to Tomcat, but also accessed via the F5 load
>>>>>>>>>>>> balancer.
>>>>>>>>>>>> Not sure where the 30 seconds comes from; perhaps a load
>>>>>>>>>>>> balancer
>>>>>>>>>>>> time
>>>>>>>>>>>> out,
>>>>>>>>>>>> as I dont see a "30" in my httpd configurations or my tomcat
>>>>>>>>>>>> server.xml
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> You can certainly look at the Apache httpd logs, and the tomcat
>>>>>>>>>>>>
>>>>>>>>>>> logs, to
>>>>>>>>>>> see if you get a request or not.
>>>>>>>>>>> In Apache httpd, you can set the loglevel individually for
>>>>>>>>>>> mod_proxy (if
>>>>>>>>>>> you are running v 2.4), and it should show something if it gets
>>>>>>>>>>> this
>>>>>>>>>>> request and forwards it to tomcat.
>>>>>>>>>>> In tomcat, you can either enable an access log (which will show
>>>>>>>>>>> if
>>>>>>>>>>> it
>>>>>>>>>>> receives this request), or you could temporarily remove/rename
>>>>>>>>>>> the
>>>>>>>>>>> /static
>>>>>>>>>>> webapp. This way, it should trigger an error "not found" which
>>>>>>>>>>> you
>>>>>>>>>>> would
>>>>>>>>>>> also see in the error log.
>>>>>>>>>>>
>>>>>>>>>>> There should be nothing between them to hinder it. We have many
>>>>>>>>>>> load
>>>>>>>>>>>
>>>>>>>>>>> balancers and this one specifically you dont need to open any
>>>>>>>>>>>> firewall
>>>>>>>>>>>> requests for the specific networks the HTTP servers are on. I
>>>>>>>>>>>> did
>>>>>>>>>>>> have to
>>>>>>>>>>>> get the firewall opened up to allow me to hit
>>>>>>>>>>>> https://loadbalancer.domain.com because the VIP for "
>>>>>>>>>>>> loadbalancer.domain.com"
>>>>>>>>>>>> is in the DMZ, and my Desktop & VPN networks cannot hit it on
>>>>>>>>>>>> 80/443
>>>>>>>>>>>> without opening holes. But beyond that, any connection from the
>>>>>>>>>>>> F5 to the
>>>>>>>>>>>> HTTP Server should be 100% open bi-directional, since same
>>>>>>>>>>>> subnet..
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> But something isn't working, otherwise you would not be asking.
>>>>>>>>>>>>
>>>>>>>>>>> So,
>>>>>>>>>>>
>>>>>>>>>>> a) hitting the tomcat webapps through httpd seems to be working
>>>>>>>>>>> fine
>>>>>>>>>>> (browser -> httpd:23270 -> tomcat:18080 -> webapp or static)
>>>>>>>>>>>
>>>>>>>>>>> b) hitting a non-proxied-to-tomcat resource of httpd seems to
>>>>>>>>>>> work
>>>>>>>>>>> fine
>>>>>>>>>>> too, even through the F5
>>>>>>>>>>> (browser -> F5:443 -> httpd:23270 -> html page)
>>>>>>>>>>>
>>>>>>>>>>> c) it is only when you do :
>>>>>>>>>>> (browser -> F5:443 -> httpd:23270 -> tomcat:18080 -> webapp
>>>>>>>>>>> or
>>>>>>>>>>> static)
>>>>>>>>>>> that you see this issue
>>>>>>>>>>>
>>>>>>>>>>> It would really help if you looked in the logs of both httpd and
>>>>>>>>>>> tomcat,
>>>>>>>>>>> and checked for differences betweens cases a, b and c above.
>>>>>>>>>>>
>>>>>>>>>>> I believe that the F5 message with the port 23270 is a minor
>>>>>>>>>>> issue,
>>>>>>>>>>> of
>>>>>>>>>>> information disclosure by the F5, that it should not disclose.
>>>>>>>>>>>
>>>>>>>>>>> But the reason why it returns this error is obviously that in
>>>>>>>>>>> that
>>>>>>>>>>> case,
>>>>>>>>>>> it does not get a response from his request to httpd.
>>>>>>>>>>> The reason for this response not coming back to the F5 (in case c
>>>>>>>>>>> only),
>>>>>>>>>>> can be due to either httpd or tomcat. But F5 doesn't know about
>>>>>>>>>>> tomcat. So
>>>>>>>>>>> for the F5, it is httpd which is not responding. Thus,
>>>>>>>>>>> - either httpd is never getting the request from the F5
>>>>>>>>>>> (unlikely,
>>>>>>>>>>> because
>>>>>>>>>>> in b above it gets it and responds)
>>>>>>>>>>> - or httpd is getting the request from the F5, but not forwarding
>>>>>>>>>>> it to
>>>>>>>>>>> tomcat, but also not returning an immediate error response to F5
>>>>>>>>>>> (which
>>>>>>>>>>> seems also unlikely, because of a and b)
>>>>>>>>>>> - or httpd is getting the request, forwarding it to tomcat, but
>>>>>>>>>>> not
>>>>>>>>>>> getting a response from tomcat. So
>>>>>>>>>>> - either tomcat is never getting the request from httpd (but
>>>>>>>>>>> in
>>>>>>>>>>> a, it
>>>>>>>>>>> gets it)
>>>>>>>>>>> - or tomcat is getting the request from httpd, but not
>>>>>>>>>>> responding (but
>>>>>>>>>>> in a, it does)
>>>>>>>>>>> - or tomcat is getting the request and responding, but the
>>>>>>>>>>> response
>>>>>>>>>>> never gets back to httpd (but in a, it does)
>>>>>>>>>>>
>>>>>>>>>>> So if a and b and c are all accurate, there is something
>>>>>>>>>>> apparently
>>>>>>>>>>> illogical happening.
>>>>>>>>>>> This would lead to the conclusion that a and b and c cannot all
>>>>>>>>>>> be
>>>>>>>>>>> accurate.
>>>>>>>>>>>
>>>>>>>>>>> The logs.. ?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Feb 21, 2017 at 2:05 PM, Antonio S. Cofino <
>>>>>>>>>>> cofinoa@gmail.com>
>>>>>>>>>>>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Aaron, on tomcat instances change the redirectPort attributte on
>>>>>>>>>>>> the http
>>>>>>>>>>>>
>>>>>>>>>>>> conectó to the loabbalancer's port 443
>>>>>>>>>>>>>
>>>>>>>>>>>>> My guess is that your webapp has restriction rule requesting
>>>>>>>>>>>>> SSL
>>>>>>>>>>>>> con
>>>>>>>>>>>>> fidntial channel. Therefore the non-confidential to the 18080
>>>>>>>>>>>>> port from
>>>>>>>>>>>>> the
>>>>>>>>>>>>> balancer are redirected to the 23270 port, but it should be
>>>>>>>>>>>>> 443.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Antonio
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> El 21/2/2017 19:46, "Aaron Gray" <aa...@gmail.com>
>>>>>>>>>>>>> escribió:
>>>>>>>>>>>>>
>>>>>>>>>>>>> I have an application server from a vendor that comes bundled
>>>>>>>>>>>>> with an
>>>>>>>>>>>>> additional Apache Tomcat server. The webapp SelfService.war is
>>>>>>>>>>>>> vendor
>>>>>>>>>>>>> supplied too.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Here's my problem (IP's replaced to protect the innocent):
>>>>>>>>>>>>>
>>>>>>>>>>>>> networks:
>>>>>>>>>>>>> DMZ=172.x.x.x
>>>>>>>>>>>>> INTERNAL=10.x.x.x
>>>>>>>>>>>>>
>>>>>>>>>>>>> server1 https listen = 172.1.1.1:23270
>>>>>>>>>>>>> server2 https listen = 172.1.1.2:23270
>>>>>>>>>>>>> F5 load balancer hostname = loadbalancer.domain.com:443
>>>>>>>>>>>>> backend tomcat server = 10.1.1.1:18080
>>>>>>>>>>>>>
>>>>>>>>>>>>> mod_proxy configuration:
>>>>>>>>>>>>> ProxyPass /SelfService http://10.1.1.1:18080/SelfService
>>>>>>>>>>>>> ProxyPassReverse /SelfService http://10.1.1.1:18080/SelfServ
>>>>>>>>>>>>> ice
>>>>>>>>>>>>>
>>>>>>>>>>>>> When I access these DMZ webservers which mod_proxy back to
>>>>>>>>>>>>> Apache
>>>>>>>>>>>>> Tomcat
>>>>>>>>>>>>> as:
>>>>>>>>>>>>> https://172.1.1.1:23270/SelfService
>>>>>>>>>>>>> and
>>>>>>>>>>>>> https://172.1.1.2:23270/SelfService <
>>>>>>>>>>>>> https://172.1.1.1:23270/SelfService
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> They load properly. Perfectly, every time!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> When I access these DMZ webservers via the F5 load balancer (to
>>>>>>>>>>>>> which I
>>>>>>>>>>>>> dont have access to, but the network folks configure for me),
>>>>>>>>>>>>> it
>>>>>>>>>>>>> hangs.
>>>>>>>>>>>>> Eventually returns:
>>>>>>>>>>>>> https://loadbalancer.domain.com:23270/SelfService
>>>>>>>>>>>>> cant load.
>>>>>>>>>>>>>
>>>>>>>>>>>>> No idea why the URL is being re-written with the ":23270".
>>>>>>>>>>>>> I added static content to the server.xml on 10.1.1.1 (Tomcat)
>>>>>>>>>>>>> to
>>>>>>>>>>>>> test:
>>>>>>>>>>>>> <Context docBase="/path/to/tomcat/static" path="/static" />
>>>>>>>>>>>>> Then put a simple index.html in there. Accessing via the
>>>>>>>>>>>>> Apache
>>>>>>>>>>>>> Web
>>>>>>>>>>>>> Servers works fine, but if you hit it with the Load Balancer it
>>>>>>>>>>>>> once
>>>>>>>>>>>>> again
>>>>>>>>>>>>> adds the https://loadbalancer.domain.com:23270/static
>>>>>>>>>>>>>
>>>>>>>>>>>>> Do you have any thoughts? Thanks so much, I have been working
>>>>>>>>>>>>> with this
>>>>>>>>>>>>> for weeks now with no success
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>> ------------------------------------------------------------
>>>>>>>>>>> ---------
>>>>>>>>>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>>>>>>>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> ------------------------------------------------------------
>>>>>>>>> ---------
>>>>>>>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>>>>>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>> ------------------------------------------------------------
>>>>>>>> ---------
>>>>>>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>>>>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
Re: Strange URL rewrite when reverse proxy with Apache HTTP Server
Posted by "André Warnier (tomcat)" <aw...@ice-sa.com>.
To avoid getting totally confused, you may want to read this explanation first :
https://wiki.apache.org/httpd/DirectoryListings
In other words, when you send this request from the browser :
https://loadbalancer.domain.com/SelfService
The first response that /should/ come back, would be a "re-direct" response from the
(final) server, with a special HTTP header added :
Location: https://loadbalancer.domain.com/SelfService/
Then your browser should re-issue the request automatically, this time to :
https://loadbalancer.domain.com/SelfService/
(with the trailing "/" added)
Then, in a second step, what should happen is :
Your browser indicates a URL, which on the server resolves to a directory.
The server /may/ be configured in such a way, that for this kind of request, it returns an
auto-generated list of all the files in that directory, automatically.
Or it may be configured to return instead, a specific html page.
(That what you do in Apache, with the DirectoryIndex directive)
And if the server is not configured at all for that, it would probably return a response
"Forbidden", for security reasons (you probably don't want someone to be able to list the
files in that directory, unless you specifically configured the server to allow that).
Or, the server may be configured so that for such a request, it defaults to running some
standard servlet which is part of the application, and which would return the application
starting page.
That would probably be the case in your "/SelfService" tomcat webapp.
So, the 302 responses which you get in some cases, are probably normal, as explained by
all the above.
What is definitely /not/ normal, is that port that you get either in these redirect
responses, or in the error messages. Because that means that, somewhere in the chain, one
of the elements is not doing its job properly when proxying.
The point of the ProxyPassReverse directive, is precisely that : ensure that if the
"proxied to" server returns a re-direct response, the "Location:" header that is in that
redirect response would be properly edited before returning that redirect response back up
the chain, in such a way that the ultimate client would not try next, to access a URL that
leads nowhere.
And now it's really late..
On 24.02.2017 00:06, Andr� Warnier (tomcat) wrote:
> On 23.02.2017 21:17, Aaron Gray wrote:
>> Another weird thing...
>>
>> If you go to:
>> https://loadbalancer.domain.com/SelfService
>> If I am viewing the headers immedaitely I see 302 not found.
>
> 302 is not "not found". It is "Found", and it is a Redirect response.
> See : https://en.wikipedia.org/wiki/HTTP_302
> for a good explanation.
> (But stop after the second paragraph, otherwise you'll get confused again..)
>
> Then I wait
>> the 30 sec for it to totally fail and change the URL in the browser to
>> https://loadbalancer.domain.com:232700/SelfService
>>
>> If you just remove the :232700 and hit Enter, it then loads properly. See
>> immedaitly 200 OK and it is fine. Don't navigate anywhere else like to
>> https://loadbalancer.domain.com/static or else you'll have to repeat the
>> same thing as above with the :232700 and removing it, hit Enter.
>>
>> Weird.
>>
>> Of course none of this is an issue if you dont hit it with the F5 Load
>> Balancer to start.
>
> Yes. There seems to be something weird *when a Redirect response is being sent back by
> tomcat*, and has to go back through the load balancer.
> And it may be linked to the fact that the load-balancer does not seem to remove this
> :23270 port which eventually reaches the browser.
>
>
>
>>
>> On Thu, Feb 23, 2017 at 10:24 AM, Aaron Gray <aa...@gmail.com> wrote:
>>
>>> I randomly tried https://loadbalancer.domain.com/SelfService/index.html
>>>
>>> It works perfectly. No issues. I have no idea why its needed to add on
>>> the /index.html (i'm lucky this exists in the tomcat .war, as it was a
>>> guess). This is visually not appealing, but workable. Maybe I can do some
>>> url rewrite or something, IF this cant be fixed normally?
>>>
>>> On Thu, Feb 23, 2017 at 10:10 AM, Aaron Gray <aa...@gmail.com> wrote:
>>>
>>>> Okay, guys, please accept my most sincere apology on this thing. I just
>>>> figured 1 thing out.
>>>>
>>>> browser -> f5:443 -> httpd (23270, https) -> tomcat (http)
>>>> using the /static works! (see below)
>>>>
>>>> If /static/index.html wasnt specified, then it hangs, then comes back as
>>>> https://loadbalancer.domain.com:23270/static
>>>> But if you go https://loadbalancer.domain.com/static/index.html it
>>>> works!!!
>>>>
>>>> DirectoryIndex index.html
>>>> is present. because if you just go to https://webserver1:23270/static
>>>> it works fine, no need for the extra /index.html
>>>>
>>>> not sure why that is different there.
>>>>
>>>> I only found this out using curl and just happened to think of it.
>>>> So you see the difference, from the tomcat access log, in the two
>>>> variations.
>>>>
>>>> 172.18.160.66 - - [23/Feb/2017:10:00:48 -0800] "GET /static HTTP/1.1" 302
>>>> -
>>>> 172.18.160.66 - - [23/Feb/2017:10:03:04 -0800] "GET /static/index.html
>>>> HTTP/1.1" 200 32
>>>>
>>>> Still /SelfService wont work though
>>>> I cant think of a way to do /SelfService/something...
>>>> Since it is a web app and i am no developer and its all from the vendor
>>>> (BMC Software) and it never shows naything other than /SelfService in the
>>>> URL. I dont think its a frame, but once again i dont know these things. I
>>>> know they told me it is HTML5, as it was using Silverlight back in the
>>>> older release.
>>>>
>>>>
>>>> On Thu, Feb 23, 2017 at 9:22 AM, Aaron Gray <aa...@gmail.com> wrote:
>>>>
>>>>> Working with my F5 guy, we had an idea, since 80/tcp and 443/tcp were
>>>>> already open to the VIP on the F5, we simply turned of 80 -> 443 redirect
>>>>> on the F5, and then configured the F5 to use the non-HTTP port in Apache
>>>>> HTTP Server (the two backend servers). So its HTTP the entire way through,
>>>>> and the same issue persisted.
>>>>>
>>>>> browser -> F5(http) -> httpd(http port) -> tomcat(http port, nonssl)
>>>>>
>>>>> So i think i can rule out any possibility that the fact I was
>>>>> terminating SSL in Apache HTTP Server then proxying clear-text back as a
>>>>> reason. It worked fine like I said if you just go direct to the HTTP
>>>>> Server and slip the F5. And F5 works landing the /healthcheck.html (HTTP
>>>>> KeepAlive in the root, which isnt proxied), so all good there. Just cant
>>>>> /SelfService (configured w/ Proxy)
>>>>>
>>>>> thanks for your support here though.
>>>>>
>>>>> On Thu, Feb 23, 2017 at 9:12 AM, Aaron Gray <aa...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> SSLProxyEngine On
>>>>>> Was already turned on this entire time inside the ssl.conf (I include
>>>>>> it) VirtualHost section.
>>>>>>
>>>>>> I am debating turning on HTTPS in Tomcat on the backend 10.x.x.x app
>>>>>> server, and then HTTPS the whole way through and see if that makes any
>>>>>> difference. I may need to request a new firewall to be opened, which is
>>>>>> not able to open until 3/2. Gonna see.
>>>>>>
>>>>>>
>>>>>> On Thu, Feb 23, 2017 at 8:22 AM, Antonio S. Cofi�o <co...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>>
>>>>>>> On 23/02/17 12:43, Andr� Warnier (tomcat) wrote:
>>>>>>>
>>>>>>>> On 22.02.2017 19:22, Aaron Gray wrote:
>>>>>>>>
>>>>>>>>> So this is interesting.
>>>>>>>>>
>>>>>>>>> So from HTTP server #1 (172.1.1.1 example) I hit:
>>>>>>>>> https://172.1.1.1:23270/static and I see this in the HTTP log:
>>>>>>>>> 172.1.1.1 - - [22/Feb/2017:10:14:48 -0800] "GET /static/ HTTP/1.1"
>>>>>>>>> 200 32
>>>>>>>>> I see this in the Tomcat log:
>>>>>>>>> 172.1.1.1 - - [22/Feb/2017:10:14:48 -0800] "GET /static/ HTTP/1.1"
>>>>>>>>> 200 32
>>>>>>>>> ((( Yes I know they look identical, but the logformat is the same,
>>>>>>>>> so it
>>>>>>>>> comes out looking the same, but they are two diff log files totally
>>>>>>>>> )))
>>>>>>>>>
>>>>>>>>> I then hit the https://loadbalancer.domain.com/static from my Win10
>>>>>>>>> laptop
>>>>>>>>> (10.1.1.2 example)
>>>>>>>>> and I see this in the HTTP server #1 log, but NOTHING in the tomcat
>>>>>>>>> access
>>>>>>>>> log.
>>>>>>>>> 10.1.1.2 - - [22/Feb/2017:10:20:02 -0800] "-" 408 -
>>>>>>>>>
>>>>>>>>> So 408. Timeout. Hmm... Why? We KNOW that it can connect from
>>>>>>>>> http ->
>>>>>>>>> tomcat:18080 perfectly. So is it a timeout BACK to the F5? That i
>>>>>>>>> dont
>>>>>>>>> know yet.
>>>>>>>>>
>>>>>>>>
>>>>>>>> It looks as if, in that case, httpd is trying to connect to tomcat,
>>>>>>>> but
>>>>>>>> - either it /is/ connecting, and sending the request, but tomcat
>>>>>>>> never answers
>>>>>>>> - or it is connecting to something else, which isn't tomcat
>>>>>>>> (but which accepts the request and doesn't answer either)
>>>>>>>>
>>>>>>>> I am not *sure*, but I believe, that if httpd was trying to connect
>>>>>>>> to something that isn't tomcat, and is not listening on port 18080, it
>>>>>>>> would get a different error, and a different logfile message. So it would
>>>>>>>> seem that at least the TCP connection works.
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Is the F5 loadbalancer configured to make reverse proxy to a SSL http?
>>>>>>> The directive SSLProxyEngine onthe F5 loadbalencer has to be enabled.
>>>>>>> http://httpd.apache.org/docs/current/mod/mod_ssl.html#sslproxyengine
>>>>>>>
>>>>>>> If the loadbalancer it's delegating the SSL work to the server1
>>>>>>> server1 https listen = 172.1.1.1:23270
>>>>>>>
>>>>>>> then the F5 could be connecting as http regular TCP connection to a
>>>>>>> SSL TCP one and therefor the SSL handshake it's not happening.
>>>>>>>
>>>>>>> Antonio
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> Now there is still a puzzle there : when you connect as
>>>>>>>> browser --> httpd --> tomcat
>>>>>>>> then everything works : httpd connects to tomcat, sends a request,
>>>>>>>> gets a response, and returns it to the browser.
>>>>>>>>
>>>>>>>> But when you do the same as :
>>>>>>>> browser --> F5 --> httpd --> tomcat
>>>>>>>> it "does not work", and the problem seems to be at httpd --> tomat
>>>>>>>> Why would it make a difference in httpd --> tomcat, if it is either
>>>>>>>> the browser or the F5 which sends the request to httpd ?
>>>>>>>> the httpd --> tomcat connection should be the same, no matter what.
>>>>>>>>
>>>>>>>> Hmm, for now as puzzled as you are.
>>>>>>>> Maybe we have a quantum phenomenon here, where the mere fact of you
>>>>>>>> looking at the logfile, changes what happened beforehand. Just kidding.
>>>>>>>>
>>>>>>>> Informational note : the Apache access log line, is written when
>>>>>>>> Apache *has finished* processing the request (including the possible
>>>>>>>> round-trip through tomcat) and has sent the response to the client. (That's
>>>>>>>> because, for example, it can also write the time spent, and the size of the
>>>>>>>> response; so it has to know those before it can write the log line).
>>>>>>>> I presume that it is the same for tomcat.
>>>>>>>> Just mentioning this, so that you would maybe have a second look at
>>>>>>>> the tomcat access log, after a few minutes, to make sure that there really
>>>>>>>> is no corresponding line there.
>>>>>>>> It could be that tomcat actually /is/ receiving the request, and
>>>>>>>> processing it, but that it takes a long time, and that httpd tires of
>>>>>>>> waiting and already times out in the meantime. In such a case, you would
>>>>>>>> probably see an error some time later in the tomcat error log, when it is
>>>>>>>> trying to eventually return the response, and finds a closed connection
>>>>>>>> instead.
>>>>>>>> Now again, why it would do that in one case and not the other, is
>>>>>>>> still a mystery.
>>>>>>>>
>>>>>>>> Another note : in the "timeout" log line above, httpd is not showing
>>>>>>>> the request URL which triggered this timeout.
>>>>>>>> a) it may be showing some additional information in the Apache error
>>>>>>>> log
>>>>>>>> b) these exists an httpd add-on module (mod_log_forensic ?) which
>>>>>>>> could provide more details in the log, about what happens precisely
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> On Tue, Feb 21, 2017 at 3:39 PM, Andr� Warnier (tomcat) <
>>>>>>>>> aw@ice-sa.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> On 21.02.2017 23:28, Aaron Gray wrote:
>>>>>>>>>>
>>>>>>>>>> Antonio: The Tomcat server has no knowledge of the F5, or that it
>>>>>>>>>>> is
>>>>>>>>>>> being
>>>>>>>>>>> fronted by an Apache HTTP Server. I do SSL termination in Apache
>>>>>>>>>>> HTTP
>>>>>>>>>>> Server, and clear-text from HTTP to Tomcat.
>>>>>>>>>>> My redirect port for the normal HTTP listen in Tomcat is commented
>>>>>>>>>>> out.
>>>>>>>>>>> <Connector port="18080" protocol="HTTP/1.1"
>>>>>>>>>>> connectionTimeout="20000" />
>>>>>>>>>>> <!-- A "Connector" using the shared thread pool-->
>>>>>>>>>>> <!--
>>>>>>>>>>> <Connector executor="tomcatThreadPool"
>>>>>>>>>>> port="8080" protocol="HTTP/1.1"
>>>>>>>>>>> connectionTimeout="20000"
>>>>>>>>>>> redirectPort="8443" />
>>>>>>>>>>> -->
>>>>>>>>>>>
>>>>>>>>>>> Andre:
>>>>>>>>>>> The URL I am using is https://loadbalancer.domain.com
>>>>>>>>>>> It is listening on port 80 and 443, if you hit 80, internally it
>>>>>>>>>>> redirects
>>>>>>>>>>> you to 443. No SSL cert on the F5 load balancer. It simply sends
>>>>>>>>>>> the
>>>>>>>>>>> traffic to one of the two HTTP servers (round-robin, also tried
>>>>>>>>>>> persistence, no difference). The HTTP server is listening only
>>>>>>>>>>> for HTTPS
>>>>>>>>>>> on 23270/tcp.
>>>>>>>>>>> When I access these DMZ webservers via the F5 load balancer (to
>>>>>>>>>>> which I
>>>>>>>>>>> dont have access to, but the network folks configure for me), it
>>>>>>>>>>> hangs.
>>>>>>>>>>> Eventually returns: server1 https listen = 172.1.1.1:23270
>>>>>>>>>>>
>>>>>>>>>>> https://loadbalancer.domain.com:23270/SelfService
>>>>>>>>>>> cant load.
>>>>>>>>>>>
>>>>>>>>>>> No idea why the URL is being re-written with the ":23270".
>>>>>>>>>>> I added static content to the server.xml on 10.1.1.1 (Tomcat) to
>>>>>>>>>>> test:
>>>>>>>>>>> <Context docBase="/path/to/tomcat/static" path="/static" />
>>>>>>>>>>> Then put a simple index.html in there. Accessing via the Apache
>>>>>>>>>>> Web
>>>>>>>>>>> Servers works fine, but if you hit it with the Load Balancer it
>>>>>>>>>>> once
>>>>>>>>>>> again
>>>>>>>>>>> adds the https://loadbalancer.domain.com:23270/static
>>>>>>>>>>>
>>>>>>>>>>> Hitting https://loadbalancer.domain.com
>>>>>>>>>>> I see my "Hello world!" which is all that is in index.html. This
>>>>>>>>>>> is the
>>>>>>>>>>> DocumentRoot of HTTP, and *not* proxied over at this time.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> So in this case, there is no delay, and you get the Apache
>>>>>>>>>> httpd-hosted
>>>>>>>>>> "index.html" containing "Hello World. Right ?
>>>>>>>>>>
>>>>>>>>>> Only
>>>>>>>>>>
>>>>>>>>>> /SelfService and /static are proxied
>>>>>>>>>>> /static just being my test of static content, but still served up
>>>>>>>>>>> by
>>>>>>>>>>> Tomcat..
>>>>>>>>>>>
>>>>>>>>>>> It's exactly 30 seconds before the page cannot be loaded when
>>>>>>>>>>> trying
>>>>>>>>>>> anything proxied to Tomcat, but also accessed via the F5 load
>>>>>>>>>>> balancer.
>>>>>>>>>>> Not sure where the 30 seconds comes from; perhaps a load balancer
>>>>>>>>>>> time
>>>>>>>>>>> out,
>>>>>>>>>>> as I dont see a "30" in my httpd configurations or my tomcat
>>>>>>>>>>> server.xml
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> You can certainly look at the Apache httpd logs, and the tomcat
>>>>>>>>>> logs, to
>>>>>>>>>> see if you get a request or not.
>>>>>>>>>> In Apache httpd, you can set the loglevel individually for
>>>>>>>>>> mod_proxy (if
>>>>>>>>>> you are running v 2.4), and it should show something if it gets this
>>>>>>>>>> request and forwards it to tomcat.
>>>>>>>>>> In tomcat, you can either enable an access log (which will show if
>>>>>>>>>> it
>>>>>>>>>> receives this request), or you could temporarily remove/rename the
>>>>>>>>>> /static
>>>>>>>>>> webapp. This way, it should trigger an error "not found" which you
>>>>>>>>>> would
>>>>>>>>>> also see in the error log.
>>>>>>>>>>
>>>>>>>>>> There should be nothing between them to hinder it. We have many
>>>>>>>>>> load
>>>>>>>>>>
>>>>>>>>>>> balancers and this one specifically you dont need to open any
>>>>>>>>>>> firewall
>>>>>>>>>>> requests for the specific networks the HTTP servers are on. I did
>>>>>>>>>>> have to
>>>>>>>>>>> get the firewall opened up to allow me to hit
>>>>>>>>>>> https://loadbalancer.domain.com because the VIP for "
>>>>>>>>>>> loadbalancer.domain.com"
>>>>>>>>>>> is in the DMZ, and my Desktop & VPN networks cannot hit it on
>>>>>>>>>>> 80/443
>>>>>>>>>>> without opening holes. But beyond that, any connection from the
>>>>>>>>>>> F5 to the
>>>>>>>>>>> HTTP Server should be 100% open bi-directional, since same subnet..
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> But something isn't working, otherwise you would not be asking.
>>>>>>>>>> So,
>>>>>>>>>>
>>>>>>>>>> a) hitting the tomcat webapps through httpd seems to be working fine
>>>>>>>>>> (browser -> httpd:23270 -> tomcat:18080 -> webapp or static)
>>>>>>>>>>
>>>>>>>>>> b) hitting a non-proxied-to-tomcat resource of httpd seems to work
>>>>>>>>>> fine
>>>>>>>>>> too, even through the F5
>>>>>>>>>> (browser -> F5:443 -> httpd:23270 -> html page)
>>>>>>>>>>
>>>>>>>>>> c) it is only when you do :
>>>>>>>>>> (browser -> F5:443 -> httpd:23270 -> tomcat:18080 -> webapp or
>>>>>>>>>> static)
>>>>>>>>>> that you see this issue
>>>>>>>>>>
>>>>>>>>>> It would really help if you looked in the logs of both httpd and
>>>>>>>>>> tomcat,
>>>>>>>>>> and checked for differences betweens cases a, b and c above.
>>>>>>>>>>
>>>>>>>>>> I believe that the F5 message with the port 23270 is a minor issue,
>>>>>>>>>> of
>>>>>>>>>> information disclosure by the F5, that it should not disclose.
>>>>>>>>>>
>>>>>>>>>> But the reason why it returns this error is obviously that in that
>>>>>>>>>> case,
>>>>>>>>>> it does not get a response from his request to httpd.
>>>>>>>>>> The reason for this response not coming back to the F5 (in case c
>>>>>>>>>> only),
>>>>>>>>>> can be due to either httpd or tomcat. But F5 doesn't know about
>>>>>>>>>> tomcat. So
>>>>>>>>>> for the F5, it is httpd which is not responding. Thus,
>>>>>>>>>> - either httpd is never getting the request from the F5 (unlikely,
>>>>>>>>>> because
>>>>>>>>>> in b above it gets it and responds)
>>>>>>>>>> - or httpd is getting the request from the F5, but not forwarding
>>>>>>>>>> it to
>>>>>>>>>> tomcat, but also not returning an immediate error response to F5
>>>>>>>>>> (which
>>>>>>>>>> seems also unlikely, because of a and b)
>>>>>>>>>> - or httpd is getting the request, forwarding it to tomcat, but not
>>>>>>>>>> getting a response from tomcat. So
>>>>>>>>>> - either tomcat is never getting the request from httpd (but in
>>>>>>>>>> a, it
>>>>>>>>>> gets it)
>>>>>>>>>> - or tomcat is getting the request from httpd, but not
>>>>>>>>>> responding (but
>>>>>>>>>> in a, it does)
>>>>>>>>>> - or tomcat is getting the request and responding, but the
>>>>>>>>>> response
>>>>>>>>>> never gets back to httpd (but in a, it does)
>>>>>>>>>>
>>>>>>>>>> So if a and b and c are all accurate, there is something apparently
>>>>>>>>>> illogical happening.
>>>>>>>>>> This would lead to the conclusion that a and b and c cannot all be
>>>>>>>>>> accurate.
>>>>>>>>>>
>>>>>>>>>> The logs.. ?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Tue, Feb 21, 2017 at 2:05 PM, Antonio S. Cofino <
>>>>>>>>>> cofinoa@gmail.com>
>>>>>>>>>>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Aaron, on tomcat instances change the redirectPort attributte on
>>>>>>>>>>> the http
>>>>>>>>>>>
>>>>>>>>>>>> conect� to the loabbalancer's port 443
>>>>>>>>>>>>
>>>>>>>>>>>> My guess is that your webapp has restriction rule requesting SSL
>>>>>>>>>>>> con
>>>>>>>>>>>> fidntial channel. Therefore the non-confidential to the 18080
>>>>>>>>>>>> port from
>>>>>>>>>>>> the
>>>>>>>>>>>> balancer are redirected to the 23270 port, but it should be 443.
>>>>>>>>>>>>
>>>>>>>>>>>> Antonio
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> El 21/2/2017 19:46, "Aaron Gray" <aa...@gmail.com> escribi�:
>>>>>>>>>>>>
>>>>>>>>>>>> I have an application server from a vendor that comes bundled
>>>>>>>>>>>> with an
>>>>>>>>>>>> additional Apache Tomcat server. The webapp SelfService.war is
>>>>>>>>>>>> vendor
>>>>>>>>>>>> supplied too.
>>>>>>>>>>>>
>>>>>>>>>>>> Here's my problem (IP's replaced to protect the innocent):
>>>>>>>>>>>>
>>>>>>>>>>>> networks:
>>>>>>>>>>>> DMZ=172.x.x.x
>>>>>>>>>>>> INTERNAL=10.x.x.x
>>>>>>>>>>>>
>>>>>>>>>>>> server1 https listen = 172.1.1.1:23270
>>>>>>>>>>>> server2 https listen = 172.1.1.2:23270
>>>>>>>>>>>> F5 load balancer hostname = loadbalancer.domain.com:443
>>>>>>>>>>>> backend tomcat server = 10.1.1.1:18080
>>>>>>>>>>>>
>>>>>>>>>>>> mod_proxy configuration:
>>>>>>>>>>>> ProxyPass /SelfService http://10.1.1.1:18080/SelfService
>>>>>>>>>>>> ProxyPassReverse /SelfService http://10.1.1.1:18080/SelfService
>>>>>>>>>>>>
>>>>>>>>>>>> When I access these DMZ webservers which mod_proxy back to Apache
>>>>>>>>>>>> Tomcat
>>>>>>>>>>>> as:
>>>>>>>>>>>> https://172.1.1.1:23270/SelfService
>>>>>>>>>>>> and
>>>>>>>>>>>> https://172.1.1.2:23270/SelfService <
>>>>>>>>>>>> https://172.1.1.1:23270/SelfService
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> They load properly. Perfectly, every time!
>>>>>>>>>>>>
>>>>>>>>>>>> When I access these DMZ webservers via the F5 load balancer (to
>>>>>>>>>>>> which I
>>>>>>>>>>>> dont have access to, but the network folks configure for me), it
>>>>>>>>>>>> hangs.
>>>>>>>>>>>> Eventually returns:
>>>>>>>>>>>> https://loadbalancer.domain.com:23270/SelfService
>>>>>>>>>>>> cant load.
>>>>>>>>>>>>
>>>>>>>>>>>> No idea why the URL is being re-written with the ":23270".
>>>>>>>>>>>> I added static content to the server.xml on 10.1.1.1 (Tomcat) to
>>>>>>>>>>>> test:
>>>>>>>>>>>> <Context docBase="/path/to/tomcat/static" path="/static" />
>>>>>>>>>>>> Then put a simple index.html in there. Accessing via the Apache
>>>>>>>>>>>> Web
>>>>>>>>>>>> Servers works fine, but if you hit it with the Load Balancer it
>>>>>>>>>>>> once
>>>>>>>>>>>> again
>>>>>>>>>>>> adds the https://loadbalancer.domain.com:23270/static
>>>>>>>>>>>>
>>>>>>>>>>>> Do you have any thoughts? Thanks so much, I have been working
>>>>>>>>>>>> with this
>>>>>>>>>>>> for weeks now with no success
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> ------------------------------------------------------------
>>>>>>>>>> ---------
>>>>>>>>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>>>>>>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> ---------------------------------------------------------------------
>>>>>>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>>>>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>>>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Strange URL rewrite when reverse proxy with Apache HTTP Server
Posted by "André Warnier (tomcat)" <aw...@ice-sa.com>.
On 23.02.2017 21:17, Aaron Gray wrote:
> Another weird thing...
>
> If you go to:
> https://loadbalancer.domain.com/SelfService
> If I am viewing the headers immedaitely I see 302 not found.
302 is not "not found". It is "Found", and it is a Redirect response.
See : https://en.wikipedia.org/wiki/HTTP_302
for a good explanation.
(But stop after the second paragraph, otherwise you'll get confused again..)
Then I wait
> the 30 sec for it to totally fail and change the URL in the browser to
> https://loadbalancer.domain.com:232700/SelfService
>
> If you just remove the :232700 and hit Enter, it then loads properly. See
> immedaitly 200 OK and it is fine. Don't navigate anywhere else like to
> https://loadbalancer.domain.com/static or else you'll have to repeat the
> same thing as above with the :232700 and removing it, hit Enter.
>
> Weird.
>
> Of course none of this is an issue if you dont hit it with the F5 Load
> Balancer to start.
Yes. There seems to be something weird *when a Redirect response is being sent back by
tomcat*, and has to go back through the load balancer.
And it may be linked to the fact that the load-balancer does not seem to remove this
:23270 port which eventually reaches the browser.
>
> On Thu, Feb 23, 2017 at 10:24 AM, Aaron Gray <aa...@gmail.com> wrote:
>
>> I randomly tried https://loadbalancer.domain.com/SelfService/index.html
>>
>> It works perfectly. No issues. I have no idea why its needed to add on
>> the /index.html (i'm lucky this exists in the tomcat .war, as it was a
>> guess). This is visually not appealing, but workable. Maybe I can do some
>> url rewrite or something, IF this cant be fixed normally?
>>
>> On Thu, Feb 23, 2017 at 10:10 AM, Aaron Gray <aa...@gmail.com> wrote:
>>
>>> Okay, guys, please accept my most sincere apology on this thing. I just
>>> figured 1 thing out.
>>>
>>> browser -> f5:443 -> httpd (23270, https) -> tomcat (http)
>>> using the /static works! (see below)
>>>
>>> If /static/index.html wasnt specified, then it hangs, then comes back as
>>> https://loadbalancer.domain.com:23270/static
>>> But if you go https://loadbalancer.domain.com/static/index.html it
>>> works!!!
>>>
>>> DirectoryIndex index.html
>>> is present. because if you just go to https://webserver1:23270/static
>>> it works fine, no need for the extra /index.html
>>>
>>> not sure why that is different there.
>>>
>>> I only found this out using curl and just happened to think of it.
>>> So you see the difference, from the tomcat access log, in the two
>>> variations.
>>>
>>> 172.18.160.66 - - [23/Feb/2017:10:00:48 -0800] "GET /static HTTP/1.1" 302
>>> -
>>> 172.18.160.66 - - [23/Feb/2017:10:03:04 -0800] "GET /static/index.html
>>> HTTP/1.1" 200 32
>>>
>>> Still /SelfService wont work though
>>> I cant think of a way to do /SelfService/something...
>>> Since it is a web app and i am no developer and its all from the vendor
>>> (BMC Software) and it never shows naything other than /SelfService in the
>>> URL. I dont think its a frame, but once again i dont know these things. I
>>> know they told me it is HTML5, as it was using Silverlight back in the
>>> older release.
>>>
>>>
>>> On Thu, Feb 23, 2017 at 9:22 AM, Aaron Gray <aa...@gmail.com> wrote:
>>>
>>>> Working with my F5 guy, we had an idea, since 80/tcp and 443/tcp were
>>>> already open to the VIP on the F5, we simply turned of 80 -> 443 redirect
>>>> on the F5, and then configured the F5 to use the non-HTTP port in Apache
>>>> HTTP Server (the two backend servers). So its HTTP the entire way through,
>>>> and the same issue persisted.
>>>>
>>>> browser -> F5(http) -> httpd(http port) -> tomcat(http port, nonssl)
>>>>
>>>> So i think i can rule out any possibility that the fact I was
>>>> terminating SSL in Apache HTTP Server then proxying clear-text back as a
>>>> reason. It worked fine like I said if you just go direct to the HTTP
>>>> Server and slip the F5. And F5 works landing the /healthcheck.html (HTTP
>>>> KeepAlive in the root, which isnt proxied), so all good there. Just cant
>>>> /SelfService (configured w/ Proxy)
>>>>
>>>> thanks for your support here though.
>>>>
>>>> On Thu, Feb 23, 2017 at 9:12 AM, Aaron Gray <aa...@gmail.com>
>>>> wrote:
>>>>
>>>>> SSLProxyEngine On
>>>>> Was already turned on this entire time inside the ssl.conf (I include
>>>>> it) VirtualHost section.
>>>>>
>>>>> I am debating turning on HTTPS in Tomcat on the backend 10.x.x.x app
>>>>> server, and then HTTPS the whole way through and see if that makes any
>>>>> difference. I may need to request a new firewall to be opened, which is
>>>>> not able to open until 3/2. Gonna see.
>>>>>
>>>>>
>>>>> On Thu, Feb 23, 2017 at 8:22 AM, Antonio S. Cofi�o <co...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>>
>>>>>> On 23/02/17 12:43, Andr� Warnier (tomcat) wrote:
>>>>>>
>>>>>>> On 22.02.2017 19:22, Aaron Gray wrote:
>>>>>>>
>>>>>>>> So this is interesting.
>>>>>>>>
>>>>>>>> So from HTTP server #1 (172.1.1.1 example) I hit:
>>>>>>>> https://172.1.1.1:23270/static and I see this in the HTTP log:
>>>>>>>> 172.1.1.1 - - [22/Feb/2017:10:14:48 -0800] "GET /static/ HTTP/1.1"
>>>>>>>> 200 32
>>>>>>>> I see this in the Tomcat log:
>>>>>>>> 172.1.1.1 - - [22/Feb/2017:10:14:48 -0800] "GET /static/ HTTP/1.1"
>>>>>>>> 200 32
>>>>>>>> ((( Yes I know they look identical, but the logformat is the same,
>>>>>>>> so it
>>>>>>>> comes out looking the same, but they are two diff log files totally
>>>>>>>> )))
>>>>>>>>
>>>>>>>> I then hit the https://loadbalancer.domain.com/static from my Win10
>>>>>>>> laptop
>>>>>>>> (10.1.1.2 example)
>>>>>>>> and I see this in the HTTP server #1 log, but NOTHING in the tomcat
>>>>>>>> access
>>>>>>>> log.
>>>>>>>> 10.1.1.2 - - [22/Feb/2017:10:20:02 -0800] "-" 408 -
>>>>>>>>
>>>>>>>> So 408. Timeout. Hmm... Why? We KNOW that it can connect from
>>>>>>>> http ->
>>>>>>>> tomcat:18080 perfectly. So is it a timeout BACK to the F5? That i
>>>>>>>> dont
>>>>>>>> know yet.
>>>>>>>>
>>>>>>>
>>>>>>> It looks as if, in that case, httpd is trying to connect to tomcat,
>>>>>>> but
>>>>>>> - either it /is/ connecting, and sending the request, but tomcat
>>>>>>> never answers
>>>>>>> - or it is connecting to something else, which isn't tomcat
>>>>>>> (but which accepts the request and doesn't answer either)
>>>>>>>
>>>>>>> I am not *sure*, but I believe, that if httpd was trying to connect
>>>>>>> to something that isn't tomcat, and is not listening on port 18080, it
>>>>>>> would get a different error, and a different logfile message. So it would
>>>>>>> seem that at least the TCP connection works.
>>>>>>>
>>>>>>
>>>>>>
>>>>>> Is the F5 loadbalancer configured to make reverse proxy to a SSL http?
>>>>>> The directive SSLProxyEngine onthe F5 loadbalencer has to be enabled.
>>>>>> http://httpd.apache.org/docs/current/mod/mod_ssl.html#sslproxyengine
>>>>>>
>>>>>> If the loadbalancer it's delegating the SSL work to the server1
>>>>>> server1 https listen = 172.1.1.1:23270
>>>>>>
>>>>>> then the F5 could be connecting as http regular TCP connection to a
>>>>>> SSL TCP one and therefor the SSL handshake it's not happening.
>>>>>>
>>>>>> Antonio
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Now there is still a puzzle there : when you connect as
>>>>>>> browser --> httpd --> tomcat
>>>>>>> then everything works : httpd connects to tomcat, sends a request,
>>>>>>> gets a response, and returns it to the browser.
>>>>>>>
>>>>>>> But when you do the same as :
>>>>>>> browser --> F5 --> httpd --> tomcat
>>>>>>> it "does not work", and the problem seems to be at httpd --> tomat
>>>>>>> Why would it make a difference in httpd --> tomcat, if it is either
>>>>>>> the browser or the F5 which sends the request to httpd ?
>>>>>>> the httpd --> tomcat connection should be the same, no matter what.
>>>>>>>
>>>>>>> Hmm, for now as puzzled as you are.
>>>>>>> Maybe we have a quantum phenomenon here, where the mere fact of you
>>>>>>> looking at the logfile, changes what happened beforehand. Just kidding.
>>>>>>>
>>>>>>> Informational note : the Apache access log line, is written when
>>>>>>> Apache *has finished* processing the request (including the possible
>>>>>>> round-trip through tomcat) and has sent the response to the client. (That's
>>>>>>> because, for example, it can also write the time spent, and the size of the
>>>>>>> response; so it has to know those before it can write the log line).
>>>>>>> I presume that it is the same for tomcat.
>>>>>>> Just mentioning this, so that you would maybe have a second look at
>>>>>>> the tomcat access log, after a few minutes, to make sure that there really
>>>>>>> is no corresponding line there.
>>>>>>> It could be that tomcat actually /is/ receiving the request, and
>>>>>>> processing it, but that it takes a long time, and that httpd tires of
>>>>>>> waiting and already times out in the meantime. In such a case, you would
>>>>>>> probably see an error some time later in the tomcat error log, when it is
>>>>>>> trying to eventually return the response, and finds a closed connection
>>>>>>> instead.
>>>>>>> Now again, why it would do that in one case and not the other, is
>>>>>>> still a mystery.
>>>>>>>
>>>>>>> Another note : in the "timeout" log line above, httpd is not showing
>>>>>>> the request URL which triggered this timeout.
>>>>>>> a) it may be showing some additional information in the Apache error
>>>>>>> log
>>>>>>> b) these exists an httpd add-on module (mod_log_forensic ?) which
>>>>>>> could provide more details in the log, about what happens precisely
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> On Tue, Feb 21, 2017 at 3:39 PM, Andr� Warnier (tomcat) <
>>>>>>>> aw@ice-sa.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> On 21.02.2017 23:28, Aaron Gray wrote:
>>>>>>>>>
>>>>>>>>> Antonio: The Tomcat server has no knowledge of the F5, or that it
>>>>>>>>>> is
>>>>>>>>>> being
>>>>>>>>>> fronted by an Apache HTTP Server. I do SSL termination in Apache
>>>>>>>>>> HTTP
>>>>>>>>>> Server, and clear-text from HTTP to Tomcat.
>>>>>>>>>> My redirect port for the normal HTTP listen in Tomcat is commented
>>>>>>>>>> out.
>>>>>>>>>> <Connector port="18080" protocol="HTTP/1.1"
>>>>>>>>>> connectionTimeout="20000" />
>>>>>>>>>> <!-- A "Connector" using the shared thread pool-->
>>>>>>>>>> <!--
>>>>>>>>>> <Connector executor="tomcatThreadPool"
>>>>>>>>>> port="8080" protocol="HTTP/1.1"
>>>>>>>>>> connectionTimeout="20000"
>>>>>>>>>> redirectPort="8443" />
>>>>>>>>>> -->
>>>>>>>>>>
>>>>>>>>>> Andre:
>>>>>>>>>> The URL I am using is https://loadbalancer.domain.com
>>>>>>>>>> It is listening on port 80 and 443, if you hit 80, internally it
>>>>>>>>>> redirects
>>>>>>>>>> you to 443. No SSL cert on the F5 load balancer. It simply sends
>>>>>>>>>> the
>>>>>>>>>> traffic to one of the two HTTP servers (round-robin, also tried
>>>>>>>>>> persistence, no difference). The HTTP server is listening only
>>>>>>>>>> for HTTPS
>>>>>>>>>> on 23270/tcp.
>>>>>>>>>> When I access these DMZ webservers via the F5 load balancer (to
>>>>>>>>>> which I
>>>>>>>>>> dont have access to, but the network folks configure for me), it
>>>>>>>>>> hangs.
>>>>>>>>>> Eventually returns: server1 https listen = 172.1.1.1:23270
>>>>>>>>>>
>>>>>>>>>> https://loadbalancer.domain.com:23270/SelfService
>>>>>>>>>> cant load.
>>>>>>>>>>
>>>>>>>>>> No idea why the URL is being re-written with the ":23270".
>>>>>>>>>> I added static content to the server.xml on 10.1.1.1 (Tomcat) to
>>>>>>>>>> test:
>>>>>>>>>> <Context docBase="/path/to/tomcat/static" path="/static" />
>>>>>>>>>> Then put a simple index.html in there. Accessing via the Apache
>>>>>>>>>> Web
>>>>>>>>>> Servers works fine, but if you hit it with the Load Balancer it
>>>>>>>>>> once
>>>>>>>>>> again
>>>>>>>>>> adds the https://loadbalancer.domain.com:23270/static
>>>>>>>>>>
>>>>>>>>>> Hitting https://loadbalancer.domain.com
>>>>>>>>>> I see my "Hello world!" which is all that is in index.html. This
>>>>>>>>>> is the
>>>>>>>>>> DocumentRoot of HTTP, and *not* proxied over at this time.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> So in this case, there is no delay, and you get the Apache
>>>>>>>>> httpd-hosted
>>>>>>>>> "index.html" containing "Hello World. Right ?
>>>>>>>>>
>>>>>>>>> Only
>>>>>>>>>
>>>>>>>>> /SelfService and /static are proxied
>>>>>>>>>> /static just being my test of static content, but still served up
>>>>>>>>>> by
>>>>>>>>>> Tomcat..
>>>>>>>>>>
>>>>>>>>>> It's exactly 30 seconds before the page cannot be loaded when
>>>>>>>>>> trying
>>>>>>>>>> anything proxied to Tomcat, but also accessed via the F5 load
>>>>>>>>>> balancer.
>>>>>>>>>> Not sure where the 30 seconds comes from; perhaps a load balancer
>>>>>>>>>> time
>>>>>>>>>> out,
>>>>>>>>>> as I dont see a "30" in my httpd configurations or my tomcat
>>>>>>>>>> server.xml
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> You can certainly look at the Apache httpd logs, and the tomcat
>>>>>>>>> logs, to
>>>>>>>>> see if you get a request or not.
>>>>>>>>> In Apache httpd, you can set the loglevel individually for
>>>>>>>>> mod_proxy (if
>>>>>>>>> you are running v 2.4), and it should show something if it gets this
>>>>>>>>> request and forwards it to tomcat.
>>>>>>>>> In tomcat, you can either enable an access log (which will show if
>>>>>>>>> it
>>>>>>>>> receives this request), or you could temporarily remove/rename the
>>>>>>>>> /static
>>>>>>>>> webapp. This way, it should trigger an error "not found" which you
>>>>>>>>> would
>>>>>>>>> also see in the error log.
>>>>>>>>>
>>>>>>>>> There should be nothing between them to hinder it. We have many
>>>>>>>>> load
>>>>>>>>>
>>>>>>>>>> balancers and this one specifically you dont need to open any
>>>>>>>>>> firewall
>>>>>>>>>> requests for the specific networks the HTTP servers are on. I did
>>>>>>>>>> have to
>>>>>>>>>> get the firewall opened up to allow me to hit
>>>>>>>>>> https://loadbalancer.domain.com because the VIP for "
>>>>>>>>>> loadbalancer.domain.com"
>>>>>>>>>> is in the DMZ, and my Desktop & VPN networks cannot hit it on
>>>>>>>>>> 80/443
>>>>>>>>>> without opening holes. But beyond that, any connection from the
>>>>>>>>>> F5 to the
>>>>>>>>>> HTTP Server should be 100% open bi-directional, since same subnet..
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> But something isn't working, otherwise you would not be asking.
>>>>>>>>> So,
>>>>>>>>>
>>>>>>>>> a) hitting the tomcat webapps through httpd seems to be working fine
>>>>>>>>> (browser -> httpd:23270 -> tomcat:18080 -> webapp or static)
>>>>>>>>>
>>>>>>>>> b) hitting a non-proxied-to-tomcat resource of httpd seems to work
>>>>>>>>> fine
>>>>>>>>> too, even through the F5
>>>>>>>>> (browser -> F5:443 -> httpd:23270 -> html page)
>>>>>>>>>
>>>>>>>>> c) it is only when you do :
>>>>>>>>> (browser -> F5:443 -> httpd:23270 -> tomcat:18080 -> webapp or
>>>>>>>>> static)
>>>>>>>>> that you see this issue
>>>>>>>>>
>>>>>>>>> It would really help if you looked in the logs of both httpd and
>>>>>>>>> tomcat,
>>>>>>>>> and checked for differences betweens cases a, b and c above.
>>>>>>>>>
>>>>>>>>> I believe that the F5 message with the port 23270 is a minor issue,
>>>>>>>>> of
>>>>>>>>> information disclosure by the F5, that it should not disclose.
>>>>>>>>>
>>>>>>>>> But the reason why it returns this error is obviously that in that
>>>>>>>>> case,
>>>>>>>>> it does not get a response from his request to httpd.
>>>>>>>>> The reason for this response not coming back to the F5 (in case c
>>>>>>>>> only),
>>>>>>>>> can be due to either httpd or tomcat. But F5 doesn't know about
>>>>>>>>> tomcat. So
>>>>>>>>> for the F5, it is httpd which is not responding. Thus,
>>>>>>>>> - either httpd is never getting the request from the F5 (unlikely,
>>>>>>>>> because
>>>>>>>>> in b above it gets it and responds)
>>>>>>>>> - or httpd is getting the request from the F5, but not forwarding
>>>>>>>>> it to
>>>>>>>>> tomcat, but also not returning an immediate error response to F5
>>>>>>>>> (which
>>>>>>>>> seems also unlikely, because of a and b)
>>>>>>>>> - or httpd is getting the request, forwarding it to tomcat, but not
>>>>>>>>> getting a response from tomcat. So
>>>>>>>>> - either tomcat is never getting the request from httpd (but in
>>>>>>>>> a, it
>>>>>>>>> gets it)
>>>>>>>>> - or tomcat is getting the request from httpd, but not
>>>>>>>>> responding (but
>>>>>>>>> in a, it does)
>>>>>>>>> - or tomcat is getting the request and responding, but the
>>>>>>>>> response
>>>>>>>>> never gets back to httpd (but in a, it does)
>>>>>>>>>
>>>>>>>>> So if a and b and c are all accurate, there is something apparently
>>>>>>>>> illogical happening.
>>>>>>>>> This would lead to the conclusion that a and b and c cannot all be
>>>>>>>>> accurate.
>>>>>>>>>
>>>>>>>>> The logs.. ?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, Feb 21, 2017 at 2:05 PM, Antonio S. Cofino <
>>>>>>>>> cofinoa@gmail.com>
>>>>>>>>>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Aaron, on tomcat instances change the redirectPort attributte on
>>>>>>>>>> the http
>>>>>>>>>>
>>>>>>>>>>> conect� to the loabbalancer's port 443
>>>>>>>>>>>
>>>>>>>>>>> My guess is that your webapp has restriction rule requesting SSL
>>>>>>>>>>> con
>>>>>>>>>>> fidntial channel. Therefore the non-confidential to the 18080
>>>>>>>>>>> port from
>>>>>>>>>>> the
>>>>>>>>>>> balancer are redirected to the 23270 port, but it should be 443.
>>>>>>>>>>>
>>>>>>>>>>> Antonio
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> El 21/2/2017 19:46, "Aaron Gray" <aa...@gmail.com> escribi�:
>>>>>>>>>>>
>>>>>>>>>>> I have an application server from a vendor that comes bundled
>>>>>>>>>>> with an
>>>>>>>>>>> additional Apache Tomcat server. The webapp SelfService.war is
>>>>>>>>>>> vendor
>>>>>>>>>>> supplied too.
>>>>>>>>>>>
>>>>>>>>>>> Here's my problem (IP's replaced to protect the innocent):
>>>>>>>>>>>
>>>>>>>>>>> networks:
>>>>>>>>>>> DMZ=172.x.x.x
>>>>>>>>>>> INTERNAL=10.x.x.x
>>>>>>>>>>>
>>>>>>>>>>> server1 https listen = 172.1.1.1:23270
>>>>>>>>>>> server2 https listen = 172.1.1.2:23270
>>>>>>>>>>> F5 load balancer hostname = loadbalancer.domain.com:443
>>>>>>>>>>> backend tomcat server = 10.1.1.1:18080
>>>>>>>>>>>
>>>>>>>>>>> mod_proxy configuration:
>>>>>>>>>>> ProxyPass /SelfService http://10.1.1.1:18080/SelfService
>>>>>>>>>>> ProxyPassReverse /SelfService http://10.1.1.1:18080/SelfService
>>>>>>>>>>>
>>>>>>>>>>> When I access these DMZ webservers which mod_proxy back to Apache
>>>>>>>>>>> Tomcat
>>>>>>>>>>> as:
>>>>>>>>>>> https://172.1.1.1:23270/SelfService
>>>>>>>>>>> and
>>>>>>>>>>> https://172.1.1.2:23270/SelfService <
>>>>>>>>>>> https://172.1.1.1:23270/SelfService
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> They load properly. Perfectly, every time!
>>>>>>>>>>>
>>>>>>>>>>> When I access these DMZ webservers via the F5 load balancer (to
>>>>>>>>>>> which I
>>>>>>>>>>> dont have access to, but the network folks configure for me), it
>>>>>>>>>>> hangs.
>>>>>>>>>>> Eventually returns:
>>>>>>>>>>> https://loadbalancer.domain.com:23270/SelfService
>>>>>>>>>>> cant load.
>>>>>>>>>>>
>>>>>>>>>>> No idea why the URL is being re-written with the ":23270".
>>>>>>>>>>> I added static content to the server.xml on 10.1.1.1 (Tomcat) to
>>>>>>>>>>> test:
>>>>>>>>>>> <Context docBase="/path/to/tomcat/static" path="/static" />
>>>>>>>>>>> Then put a simple index.html in there. Accessing via the Apache
>>>>>>>>>>> Web
>>>>>>>>>>> Servers works fine, but if you hit it with the Load Balancer it
>>>>>>>>>>> once
>>>>>>>>>>> again
>>>>>>>>>>> adds the https://loadbalancer.domain.com:23270/static
>>>>>>>>>>>
>>>>>>>>>>> Do you have any thoughts? Thanks so much, I have been working
>>>>>>>>>>> with this
>>>>>>>>>>> for weeks now with no success
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> ------------------------------------------------------------
>>>>>>>>> ---------
>>>>>>>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>>>>>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>>>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Strange URL rewrite when reverse proxy with Apache HTTP Server
Posted by Aaron Gray <aa...@gmail.com>.
Another weird thing...
If you go to:
https://loadbalancer.domain.com/SelfService
If I am viewing the headers immedaitely I see 302 not found. Then I wait
the 30 sec for it to totally fail and change the URL in the browser to
https://loadbalancer.domain.com:232700/SelfService
If you just remove the :232700 and hit Enter, it then loads properly. See
immedaitly 200 OK and it is fine. Don't navigate anywhere else like to
https://loadbalancer.domain.com/static or else you'll have to repeat the
same thing as above with the :232700 and removing it, hit Enter.
Weird.
Of course none of this is an issue if you dont hit it with the F5 Load
Balancer to start.
On Thu, Feb 23, 2017 at 10:24 AM, Aaron Gray <aa...@gmail.com> wrote:
> I randomly tried https://loadbalancer.domain.com/SelfService/index.html
>
> It works perfectly. No issues. I have no idea why its needed to add on
> the /index.html (i'm lucky this exists in the tomcat .war, as it was a
> guess). This is visually not appealing, but workable. Maybe I can do some
> url rewrite or something, IF this cant be fixed normally?
>
> On Thu, Feb 23, 2017 at 10:10 AM, Aaron Gray <aa...@gmail.com> wrote:
>
>> Okay, guys, please accept my most sincere apology on this thing. I just
>> figured 1 thing out.
>>
>> browser -> f5:443 -> httpd (23270, https) -> tomcat (http)
>> using the /static works! (see below)
>>
>> If /static/index.html wasnt specified, then it hangs, then comes back as
>> https://loadbalancer.domain.com:23270/static
>> But if you go https://loadbalancer.domain.com/static/index.html it
>> works!!!
>>
>> DirectoryIndex index.html
>> is present. because if you just go to https://webserver1:23270/static
>> it works fine, no need for the extra /index.html
>>
>> not sure why that is different there.
>>
>> I only found this out using curl and just happened to think of it.
>> So you see the difference, from the tomcat access log, in the two
>> variations.
>>
>> 172.18.160.66 - - [23/Feb/2017:10:00:48 -0800] "GET /static HTTP/1.1" 302
>> -
>> 172.18.160.66 - - [23/Feb/2017:10:03:04 -0800] "GET /static/index.html
>> HTTP/1.1" 200 32
>>
>> Still /SelfService wont work though
>> I cant think of a way to do /SelfService/something...
>> Since it is a web app and i am no developer and its all from the vendor
>> (BMC Software) and it never shows naything other than /SelfService in the
>> URL. I dont think its a frame, but once again i dont know these things. I
>> know they told me it is HTML5, as it was using Silverlight back in the
>> older release.
>>
>>
>> On Thu, Feb 23, 2017 at 9:22 AM, Aaron Gray <aa...@gmail.com> wrote:
>>
>>> Working with my F5 guy, we had an idea, since 80/tcp and 443/tcp were
>>> already open to the VIP on the F5, we simply turned of 80 -> 443 redirect
>>> on the F5, and then configured the F5 to use the non-HTTP port in Apache
>>> HTTP Server (the two backend servers). So its HTTP the entire way through,
>>> and the same issue persisted.
>>>
>>> browser -> F5(http) -> httpd(http port) -> tomcat(http port, nonssl)
>>>
>>> So i think i can rule out any possibility that the fact I was
>>> terminating SSL in Apache HTTP Server then proxying clear-text back as a
>>> reason. It worked fine like I said if you just go direct to the HTTP
>>> Server and slip the F5. And F5 works landing the /healthcheck.html (HTTP
>>> KeepAlive in the root, which isnt proxied), so all good there. Just cant
>>> /SelfService (configured w/ Proxy)
>>>
>>> thanks for your support here though.
>>>
>>> On Thu, Feb 23, 2017 at 9:12 AM, Aaron Gray <aa...@gmail.com>
>>> wrote:
>>>
>>>> SSLProxyEngine On
>>>> Was already turned on this entire time inside the ssl.conf (I include
>>>> it) VirtualHost section.
>>>>
>>>> I am debating turning on HTTPS in Tomcat on the backend 10.x.x.x app
>>>> server, and then HTTPS the whole way through and see if that makes any
>>>> difference. I may need to request a new firewall to be opened, which is
>>>> not able to open until 3/2. Gonna see.
>>>>
>>>>
>>>> On Thu, Feb 23, 2017 at 8:22 AM, Antonio S. Cofiño <co...@gmail.com>
>>>> wrote:
>>>>
>>>>>
>>>>> On 23/02/17 12:43, André Warnier (tomcat) wrote:
>>>>>
>>>>>> On 22.02.2017 19:22, Aaron Gray wrote:
>>>>>>
>>>>>>> So this is interesting.
>>>>>>>
>>>>>>> So from HTTP server #1 (172.1.1.1 example) I hit:
>>>>>>> https://172.1.1.1:23270/static and I see this in the HTTP log:
>>>>>>> 172.1.1.1 - - [22/Feb/2017:10:14:48 -0800] "GET /static/ HTTP/1.1"
>>>>>>> 200 32
>>>>>>> I see this in the Tomcat log:
>>>>>>> 172.1.1.1 - - [22/Feb/2017:10:14:48 -0800] "GET /static/ HTTP/1.1"
>>>>>>> 200 32
>>>>>>> ((( Yes I know they look identical, but the logformat is the same,
>>>>>>> so it
>>>>>>> comes out looking the same, but they are two diff log files totally
>>>>>>> )))
>>>>>>>
>>>>>>> I then hit the https://loadbalancer.domain.com/static from my Win10
>>>>>>> laptop
>>>>>>> (10.1.1.2 example)
>>>>>>> and I see this in the HTTP server #1 log, but NOTHING in the tomcat
>>>>>>> access
>>>>>>> log.
>>>>>>> 10.1.1.2 - - [22/Feb/2017:10:20:02 -0800] "-" 408 -
>>>>>>>
>>>>>>> So 408. Timeout. Hmm... Why? We KNOW that it can connect from
>>>>>>> http ->
>>>>>>> tomcat:18080 perfectly. So is it a timeout BACK to the F5? That i
>>>>>>> dont
>>>>>>> know yet.
>>>>>>>
>>>>>>
>>>>>> It looks as if, in that case, httpd is trying to connect to tomcat,
>>>>>> but
>>>>>> - either it /is/ connecting, and sending the request, but tomcat
>>>>>> never answers
>>>>>> - or it is connecting to something else, which isn't tomcat
>>>>>> (but which accepts the request and doesn't answer either)
>>>>>>
>>>>>> I am not *sure*, but I believe, that if httpd was trying to connect
>>>>>> to something that isn't tomcat, and is not listening on port 18080, it
>>>>>> would get a different error, and a different logfile message. So it would
>>>>>> seem that at least the TCP connection works.
>>>>>>
>>>>>
>>>>>
>>>>> Is the F5 loadbalancer configured to make reverse proxy to a SSL http?
>>>>> The directive SSLProxyEngine onthe F5 loadbalencer has to be enabled.
>>>>> http://httpd.apache.org/docs/current/mod/mod_ssl.html#sslproxyengine
>>>>>
>>>>> If the loadbalancer it's delegating the SSL work to the server1
>>>>> server1 https listen = 172.1.1.1:23270
>>>>>
>>>>> then the F5 could be connecting as http regular TCP connection to a
>>>>> SSL TCP one and therefor the SSL handshake it's not happening.
>>>>>
>>>>> Antonio
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> Now there is still a puzzle there : when you connect as
>>>>>> browser --> httpd --> tomcat
>>>>>> then everything works : httpd connects to tomcat, sends a request,
>>>>>> gets a response, and returns it to the browser.
>>>>>>
>>>>>> But when you do the same as :
>>>>>> browser --> F5 --> httpd --> tomcat
>>>>>> it "does not work", and the problem seems to be at httpd --> tomat
>>>>>> Why would it make a difference in httpd --> tomcat, if it is either
>>>>>> the browser or the F5 which sends the request to httpd ?
>>>>>> the httpd --> tomcat connection should be the same, no matter what.
>>>>>>
>>>>>> Hmm, for now as puzzled as you are.
>>>>>> Maybe we have a quantum phenomenon here, where the mere fact of you
>>>>>> looking at the logfile, changes what happened beforehand. Just kidding.
>>>>>>
>>>>>> Informational note : the Apache access log line, is written when
>>>>>> Apache *has finished* processing the request (including the possible
>>>>>> round-trip through tomcat) and has sent the response to the client. (That's
>>>>>> because, for example, it can also write the time spent, and the size of the
>>>>>> response; so it has to know those before it can write the log line).
>>>>>> I presume that it is the same for tomcat.
>>>>>> Just mentioning this, so that you would maybe have a second look at
>>>>>> the tomcat access log, after a few minutes, to make sure that there really
>>>>>> is no corresponding line there.
>>>>>> It could be that tomcat actually /is/ receiving the request, and
>>>>>> processing it, but that it takes a long time, and that httpd tires of
>>>>>> waiting and already times out in the meantime. In such a case, you would
>>>>>> probably see an error some time later in the tomcat error log, when it is
>>>>>> trying to eventually return the response, and finds a closed connection
>>>>>> instead.
>>>>>> Now again, why it would do that in one case and not the other, is
>>>>>> still a mystery.
>>>>>>
>>>>>> Another note : in the "timeout" log line above, httpd is not showing
>>>>>> the request URL which triggered this timeout.
>>>>>> a) it may be showing some additional information in the Apache error
>>>>>> log
>>>>>> b) these exists an httpd add-on module (mod_log_forensic ?) which
>>>>>> could provide more details in the log, about what happens precisely
>>>>>>
>>>>>>
>>>>>>
>>>>>>> On Tue, Feb 21, 2017 at 3:39 PM, André Warnier (tomcat) <
>>>>>>> aw@ice-sa.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>> On 21.02.2017 23:28, Aaron Gray wrote:
>>>>>>>>
>>>>>>>> Antonio: The Tomcat server has no knowledge of the F5, or that it
>>>>>>>>> is
>>>>>>>>> being
>>>>>>>>> fronted by an Apache HTTP Server. I do SSL termination in Apache
>>>>>>>>> HTTP
>>>>>>>>> Server, and clear-text from HTTP to Tomcat.
>>>>>>>>> My redirect port for the normal HTTP listen in Tomcat is commented
>>>>>>>>> out.
>>>>>>>>> <Connector port="18080" protocol="HTTP/1.1"
>>>>>>>>> connectionTimeout="20000" />
>>>>>>>>> <!-- A "Connector" using the shared thread pool-->
>>>>>>>>> <!--
>>>>>>>>> <Connector executor="tomcatThreadPool"
>>>>>>>>> port="8080" protocol="HTTP/1.1"
>>>>>>>>> connectionTimeout="20000"
>>>>>>>>> redirectPort="8443" />
>>>>>>>>> -->
>>>>>>>>>
>>>>>>>>> Andre:
>>>>>>>>> The URL I am using is https://loadbalancer.domain.com
>>>>>>>>> It is listening on port 80 and 443, if you hit 80, internally it
>>>>>>>>> redirects
>>>>>>>>> you to 443. No SSL cert on the F5 load balancer. It simply sends
>>>>>>>>> the
>>>>>>>>> traffic to one of the two HTTP servers (round-robin, also tried
>>>>>>>>> persistence, no difference). The HTTP server is listening only
>>>>>>>>> for HTTPS
>>>>>>>>> on 23270/tcp.
>>>>>>>>> When I access these DMZ webservers via the F5 load balancer (to
>>>>>>>>> which I
>>>>>>>>> dont have access to, but the network folks configure for me), it
>>>>>>>>> hangs.
>>>>>>>>> Eventually returns: server1 https listen = 172.1.1.1:23270
>>>>>>>>>
>>>>>>>>> https://loadbalancer.domain.com:23270/SelfService
>>>>>>>>> cant load.
>>>>>>>>>
>>>>>>>>> No idea why the URL is being re-written with the ":23270".
>>>>>>>>> I added static content to the server.xml on 10.1.1.1 (Tomcat) to
>>>>>>>>> test:
>>>>>>>>> <Context docBase="/path/to/tomcat/static" path="/static" />
>>>>>>>>> Then put a simple index.html in there. Accessing via the Apache
>>>>>>>>> Web
>>>>>>>>> Servers works fine, but if you hit it with the Load Balancer it
>>>>>>>>> once
>>>>>>>>> again
>>>>>>>>> adds the https://loadbalancer.domain.com:23270/static
>>>>>>>>>
>>>>>>>>> Hitting https://loadbalancer.domain.com
>>>>>>>>> I see my "Hello world!" which is all that is in index.html. This
>>>>>>>>> is the
>>>>>>>>> DocumentRoot of HTTP, and *not* proxied over at this time.
>>>>>>>>>
>>>>>>>>>
>>>>>>>> So in this case, there is no delay, and you get the Apache
>>>>>>>> httpd-hosted
>>>>>>>> "index.html" containing "Hello World. Right ?
>>>>>>>>
>>>>>>>> Only
>>>>>>>>
>>>>>>>> /SelfService and /static are proxied
>>>>>>>>> /static just being my test of static content, but still served up
>>>>>>>>> by
>>>>>>>>> Tomcat..
>>>>>>>>>
>>>>>>>>> It's exactly 30 seconds before the page cannot be loaded when
>>>>>>>>> trying
>>>>>>>>> anything proxied to Tomcat, but also accessed via the F5 load
>>>>>>>>> balancer.
>>>>>>>>> Not sure where the 30 seconds comes from; perhaps a load balancer
>>>>>>>>> time
>>>>>>>>> out,
>>>>>>>>> as I dont see a "30" in my httpd configurations or my tomcat
>>>>>>>>> server.xml
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> You can certainly look at the Apache httpd logs, and the tomcat
>>>>>>>> logs, to
>>>>>>>> see if you get a request or not.
>>>>>>>> In Apache httpd, you can set the loglevel individually for
>>>>>>>> mod_proxy (if
>>>>>>>> you are running v 2.4), and it should show something if it gets this
>>>>>>>> request and forwards it to tomcat.
>>>>>>>> In tomcat, you can either enable an access log (which will show if
>>>>>>>> it
>>>>>>>> receives this request), or you could temporarily remove/rename the
>>>>>>>> /static
>>>>>>>> webapp. This way, it should trigger an error "not found" which you
>>>>>>>> would
>>>>>>>> also see in the error log.
>>>>>>>>
>>>>>>>> There should be nothing between them to hinder it. We have many
>>>>>>>> load
>>>>>>>>
>>>>>>>>> balancers and this one specifically you dont need to open any
>>>>>>>>> firewall
>>>>>>>>> requests for the specific networks the HTTP servers are on. I did
>>>>>>>>> have to
>>>>>>>>> get the firewall opened up to allow me to hit
>>>>>>>>> https://loadbalancer.domain.com because the VIP for "
>>>>>>>>> loadbalancer.domain.com"
>>>>>>>>> is in the DMZ, and my Desktop & VPN networks cannot hit it on
>>>>>>>>> 80/443
>>>>>>>>> without opening holes. But beyond that, any connection from the
>>>>>>>>> F5 to the
>>>>>>>>> HTTP Server should be 100% open bi-directional, since same subnet.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> But something isn't working, otherwise you would not be asking.
>>>>>>>> So,
>>>>>>>>
>>>>>>>> a) hitting the tomcat webapps through httpd seems to be working fine
>>>>>>>> (browser -> httpd:23270 -> tomcat:18080 -> webapp or static)
>>>>>>>>
>>>>>>>> b) hitting a non-proxied-to-tomcat resource of httpd seems to work
>>>>>>>> fine
>>>>>>>> too, even through the F5
>>>>>>>> (browser -> F5:443 -> httpd:23270 -> html page)
>>>>>>>>
>>>>>>>> c) it is only when you do :
>>>>>>>> (browser -> F5:443 -> httpd:23270 -> tomcat:18080 -> webapp or
>>>>>>>> static)
>>>>>>>> that you see this issue
>>>>>>>>
>>>>>>>> It would really help if you looked in the logs of both httpd and
>>>>>>>> tomcat,
>>>>>>>> and checked for differences betweens cases a, b and c above.
>>>>>>>>
>>>>>>>> I believe that the F5 message with the port 23270 is a minor issue,
>>>>>>>> of
>>>>>>>> information disclosure by the F5, that it should not disclose.
>>>>>>>>
>>>>>>>> But the reason why it returns this error is obviously that in that
>>>>>>>> case,
>>>>>>>> it does not get a response from his request to httpd.
>>>>>>>> The reason for this response not coming back to the F5 (in case c
>>>>>>>> only),
>>>>>>>> can be due to either httpd or tomcat. But F5 doesn't know about
>>>>>>>> tomcat. So
>>>>>>>> for the F5, it is httpd which is not responding. Thus,
>>>>>>>> - either httpd is never getting the request from the F5 (unlikely,
>>>>>>>> because
>>>>>>>> in b above it gets it and responds)
>>>>>>>> - or httpd is getting the request from the F5, but not forwarding
>>>>>>>> it to
>>>>>>>> tomcat, but also not returning an immediate error response to F5
>>>>>>>> (which
>>>>>>>> seems also unlikely, because of a and b)
>>>>>>>> - or httpd is getting the request, forwarding it to tomcat, but not
>>>>>>>> getting a response from tomcat. So
>>>>>>>> - either tomcat is never getting the request from httpd (but in
>>>>>>>> a, it
>>>>>>>> gets it)
>>>>>>>> - or tomcat is getting the request from httpd, but not
>>>>>>>> responding (but
>>>>>>>> in a, it does)
>>>>>>>> - or tomcat is getting the request and responding, but the
>>>>>>>> response
>>>>>>>> never gets back to httpd (but in a, it does)
>>>>>>>>
>>>>>>>> So if a and b and c are all accurate, there is something apparently
>>>>>>>> illogical happening.
>>>>>>>> This would lead to the conclusion that a and b and c cannot all be
>>>>>>>> accurate.
>>>>>>>>
>>>>>>>> The logs.. ?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Feb 21, 2017 at 2:05 PM, Antonio S. Cofino <
>>>>>>>> cofinoa@gmail.com>
>>>>>>>>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Aaron, on tomcat instances change the redirectPort attributte on
>>>>>>>>> the http
>>>>>>>>>
>>>>>>>>>> conectó to the loabbalancer's port 443
>>>>>>>>>>
>>>>>>>>>> My guess is that your webapp has restriction rule requesting SSL
>>>>>>>>>> con
>>>>>>>>>> fidntial channel. Therefore the non-confidential to the 18080
>>>>>>>>>> port from
>>>>>>>>>> the
>>>>>>>>>> balancer are redirected to the 23270 port, but it should be 443.
>>>>>>>>>>
>>>>>>>>>> Antonio
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> El 21/2/2017 19:46, "Aaron Gray" <aa...@gmail.com> escribió:
>>>>>>>>>>
>>>>>>>>>> I have an application server from a vendor that comes bundled
>>>>>>>>>> with an
>>>>>>>>>> additional Apache Tomcat server. The webapp SelfService.war is
>>>>>>>>>> vendor
>>>>>>>>>> supplied too.
>>>>>>>>>>
>>>>>>>>>> Here's my problem (IP's replaced to protect the innocent):
>>>>>>>>>>
>>>>>>>>>> networks:
>>>>>>>>>> DMZ=172.x.x.x
>>>>>>>>>> INTERNAL=10.x.x.x
>>>>>>>>>>
>>>>>>>>>> server1 https listen = 172.1.1.1:23270
>>>>>>>>>> server2 https listen = 172.1.1.2:23270
>>>>>>>>>> F5 load balancer hostname = loadbalancer.domain.com:443
>>>>>>>>>> backend tomcat server = 10.1.1.1:18080
>>>>>>>>>>
>>>>>>>>>> mod_proxy configuration:
>>>>>>>>>> ProxyPass /SelfService http://10.1.1.1:18080/SelfService
>>>>>>>>>> ProxyPassReverse /SelfService http://10.1.1.1:18080/SelfService
>>>>>>>>>>
>>>>>>>>>> When I access these DMZ webservers which mod_proxy back to Apache
>>>>>>>>>> Tomcat
>>>>>>>>>> as:
>>>>>>>>>> https://172.1.1.1:23270/SelfService
>>>>>>>>>> and
>>>>>>>>>> https://172.1.1.2:23270/SelfService <
>>>>>>>>>> https://172.1.1.1:23270/SelfService
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> They load properly. Perfectly, every time!
>>>>>>>>>>
>>>>>>>>>> When I access these DMZ webservers via the F5 load balancer (to
>>>>>>>>>> which I
>>>>>>>>>> dont have access to, but the network folks configure for me), it
>>>>>>>>>> hangs.
>>>>>>>>>> Eventually returns:
>>>>>>>>>> https://loadbalancer.domain.com:23270/SelfService
>>>>>>>>>> cant load.
>>>>>>>>>>
>>>>>>>>>> No idea why the URL is being re-written with the ":23270".
>>>>>>>>>> I added static content to the server.xml on 10.1.1.1 (Tomcat) to
>>>>>>>>>> test:
>>>>>>>>>> <Context docBase="/path/to/tomcat/static" path="/static" />
>>>>>>>>>> Then put a simple index.html in there. Accessing via the Apache
>>>>>>>>>> Web
>>>>>>>>>> Servers works fine, but if you hit it with the Load Balancer it
>>>>>>>>>> once
>>>>>>>>>> again
>>>>>>>>>> adds the https://loadbalancer.domain.com:23270/static
>>>>>>>>>>
>>>>>>>>>> Do you have any thoughts? Thanks so much, I have been working
>>>>>>>>>> with this
>>>>>>>>>> for weeks now with no success
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>> ------------------------------------------------------------
>>>>>>>> ---------
>>>>>>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>>>>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>>>>
>>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Strange URL rewrite when reverse proxy with Apache HTTP Server
Posted by Aaron Gray <aa...@gmail.com>.
I randomly tried https://loadbalancer.domain.com/SelfService/index.html
It works perfectly. No issues. I have no idea why its needed to add on
the /index.html (i'm lucky this exists in the tomcat .war, as it was a
guess). This is visually not appealing, but workable. Maybe I can do some
url rewrite or something, IF this cant be fixed normally?
On Thu, Feb 23, 2017 at 10:10 AM, Aaron Gray <aa...@gmail.com> wrote:
> Okay, guys, please accept my most sincere apology on this thing. I just
> figured 1 thing out.
>
> browser -> f5:443 -> httpd (23270, https) -> tomcat (http)
> using the /static works! (see below)
>
> If /static/index.html wasnt specified, then it hangs, then comes back as
> https://loadbalancer.domain.com:23270/static
> But if you go https://loadbalancer.domain.com/static/index.html it
> works!!!
>
> DirectoryIndex index.html
> is present. because if you just go to https://webserver1:23270/static
> it works fine, no need for the extra /index.html
>
> not sure why that is different there.
>
> I only found this out using curl and just happened to think of it.
> So you see the difference, from the tomcat access log, in the two
> variations.
>
> 172.18.160.66 - - [23/Feb/2017:10:00:48 -0800] "GET /static HTTP/1.1" 302 -
> 172.18.160.66 - - [23/Feb/2017:10:03:04 -0800] "GET /static/index.html
> HTTP/1.1" 200 32
>
> Still /SelfService wont work though
> I cant think of a way to do /SelfService/something...
> Since it is a web app and i am no developer and its all from the vendor
> (BMC Software) and it never shows naything other than /SelfService in the
> URL. I dont think its a frame, but once again i dont know these things. I
> know they told me it is HTML5, as it was using Silverlight back in the
> older release.
>
>
> On Thu, Feb 23, 2017 at 9:22 AM, Aaron Gray <aa...@gmail.com> wrote:
>
>> Working with my F5 guy, we had an idea, since 80/tcp and 443/tcp were
>> already open to the VIP on the F5, we simply turned of 80 -> 443 redirect
>> on the F5, and then configured the F5 to use the non-HTTP port in Apache
>> HTTP Server (the two backend servers). So its HTTP the entire way through,
>> and the same issue persisted.
>>
>> browser -> F5(http) -> httpd(http port) -> tomcat(http port, nonssl)
>>
>> So i think i can rule out any possibility that the fact I was terminating
>> SSL in Apache HTTP Server then proxying clear-text back as a reason. It
>> worked fine like I said if you just go direct to the HTTP Server and slip
>> the F5. And F5 works landing the /healthcheck.html (HTTP KeepAlive in the
>> root, which isnt proxied), so all good there. Just cant /SelfService
>> (configured w/ Proxy)
>>
>> thanks for your support here though.
>>
>> On Thu, Feb 23, 2017 at 9:12 AM, Aaron Gray <aa...@gmail.com> wrote:
>>
>>> SSLProxyEngine On
>>> Was already turned on this entire time inside the ssl.conf (I include
>>> it) VirtualHost section.
>>>
>>> I am debating turning on HTTPS in Tomcat on the backend 10.x.x.x app
>>> server, and then HTTPS the whole way through and see if that makes any
>>> difference. I may need to request a new firewall to be opened, which is
>>> not able to open until 3/2. Gonna see.
>>>
>>>
>>> On Thu, Feb 23, 2017 at 8:22 AM, Antonio S. Cofiño <co...@gmail.com>
>>> wrote:
>>>
>>>>
>>>> On 23/02/17 12:43, André Warnier (tomcat) wrote:
>>>>
>>>>> On 22.02.2017 19:22, Aaron Gray wrote:
>>>>>
>>>>>> So this is interesting.
>>>>>>
>>>>>> So from HTTP server #1 (172.1.1.1 example) I hit:
>>>>>> https://172.1.1.1:23270/static and I see this in the HTTP log:
>>>>>> 172.1.1.1 - - [22/Feb/2017:10:14:48 -0800] "GET /static/ HTTP/1.1"
>>>>>> 200 32
>>>>>> I see this in the Tomcat log:
>>>>>> 172.1.1.1 - - [22/Feb/2017:10:14:48 -0800] "GET /static/ HTTP/1.1"
>>>>>> 200 32
>>>>>> ((( Yes I know they look identical, but the logformat is the same, so
>>>>>> it
>>>>>> comes out looking the same, but they are two diff log files totally
>>>>>> )))
>>>>>>
>>>>>> I then hit the https://loadbalancer.domain.com/static from my Win10
>>>>>> laptop
>>>>>> (10.1.1.2 example)
>>>>>> and I see this in the HTTP server #1 log, but NOTHING in the tomcat
>>>>>> access
>>>>>> log.
>>>>>> 10.1.1.2 - - [22/Feb/2017:10:20:02 -0800] "-" 408 -
>>>>>>
>>>>>> So 408. Timeout. Hmm... Why? We KNOW that it can connect from http
>>>>>> ->
>>>>>> tomcat:18080 perfectly. So is it a timeout BACK to the F5? That i
>>>>>> dont
>>>>>> know yet.
>>>>>>
>>>>>
>>>>> It looks as if, in that case, httpd is trying to connect to tomcat, but
>>>>> - either it /is/ connecting, and sending the request, but tomcat never
>>>>> answers
>>>>> - or it is connecting to something else, which isn't tomcat
>>>>> (but which accepts the request and doesn't answer either)
>>>>>
>>>>> I am not *sure*, but I believe, that if httpd was trying to connect to
>>>>> something that isn't tomcat, and is not listening on port 18080, it would
>>>>> get a different error, and a different logfile message. So it would seem
>>>>> that at least the TCP connection works.
>>>>>
>>>>
>>>>
>>>> Is the F5 loadbalancer configured to make reverse proxy to a SSL http?
>>>> The directive SSLProxyEngine onthe F5 loadbalencer has to be enabled.
>>>> http://httpd.apache.org/docs/current/mod/mod_ssl.html#sslproxyengine
>>>>
>>>> If the loadbalancer it's delegating the SSL work to the server1
>>>> server1 https listen = 172.1.1.1:23270
>>>>
>>>> then the F5 could be connecting as http regular TCP connection to a SSL
>>>> TCP one and therefor the SSL handshake it's not happening.
>>>>
>>>> Antonio
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> Now there is still a puzzle there : when you connect as
>>>>> browser --> httpd --> tomcat
>>>>> then everything works : httpd connects to tomcat, sends a request,
>>>>> gets a response, and returns it to the browser.
>>>>>
>>>>> But when you do the same as :
>>>>> browser --> F5 --> httpd --> tomcat
>>>>> it "does not work", and the problem seems to be at httpd --> tomat
>>>>> Why would it make a difference in httpd --> tomcat, if it is either
>>>>> the browser or the F5 which sends the request to httpd ?
>>>>> the httpd --> tomcat connection should be the same, no matter what.
>>>>>
>>>>> Hmm, for now as puzzled as you are.
>>>>> Maybe we have a quantum phenomenon here, where the mere fact of you
>>>>> looking at the logfile, changes what happened beforehand. Just kidding.
>>>>>
>>>>> Informational note : the Apache access log line, is written when
>>>>> Apache *has finished* processing the request (including the possible
>>>>> round-trip through tomcat) and has sent the response to the client. (That's
>>>>> because, for example, it can also write the time spent, and the size of the
>>>>> response; so it has to know those before it can write the log line).
>>>>> I presume that it is the same for tomcat.
>>>>> Just mentioning this, so that you would maybe have a second look at
>>>>> the tomcat access log, after a few minutes, to make sure that there really
>>>>> is no corresponding line there.
>>>>> It could be that tomcat actually /is/ receiving the request, and
>>>>> processing it, but that it takes a long time, and that httpd tires of
>>>>> waiting and already times out in the meantime. In such a case, you would
>>>>> probably see an error some time later in the tomcat error log, when it is
>>>>> trying to eventually return the response, and finds a closed connection
>>>>> instead.
>>>>> Now again, why it would do that in one case and not the other, is
>>>>> still a mystery.
>>>>>
>>>>> Another note : in the "timeout" log line above, httpd is not showing
>>>>> the request URL which triggered this timeout.
>>>>> a) it may be showing some additional information in the Apache error
>>>>> log
>>>>> b) these exists an httpd add-on module (mod_log_forensic ?) which
>>>>> could provide more details in the log, about what happens precisely
>>>>>
>>>>>
>>>>>
>>>>>> On Tue, Feb 21, 2017 at 3:39 PM, André Warnier (tomcat) <
>>>>>> aw@ice-sa.com>
>>>>>> wrote:
>>>>>>
>>>>>> On 21.02.2017 23:28, Aaron Gray wrote:
>>>>>>>
>>>>>>> Antonio: The Tomcat server has no knowledge of the F5, or that it is
>>>>>>>> being
>>>>>>>> fronted by an Apache HTTP Server. I do SSL termination in Apache
>>>>>>>> HTTP
>>>>>>>> Server, and clear-text from HTTP to Tomcat.
>>>>>>>> My redirect port for the normal HTTP listen in Tomcat is commented
>>>>>>>> out.
>>>>>>>> <Connector port="18080" protocol="HTTP/1.1"
>>>>>>>> connectionTimeout="20000" />
>>>>>>>> <!-- A "Connector" using the shared thread pool-->
>>>>>>>> <!--
>>>>>>>> <Connector executor="tomcatThreadPool"
>>>>>>>> port="8080" protocol="HTTP/1.1"
>>>>>>>> connectionTimeout="20000"
>>>>>>>> redirectPort="8443" />
>>>>>>>> -->
>>>>>>>>
>>>>>>>> Andre:
>>>>>>>> The URL I am using is https://loadbalancer.domain.com
>>>>>>>> It is listening on port 80 and 443, if you hit 80, internally it
>>>>>>>> redirects
>>>>>>>> you to 443. No SSL cert on the F5 load balancer. It simply sends
>>>>>>>> the
>>>>>>>> traffic to one of the two HTTP servers (round-robin, also tried
>>>>>>>> persistence, no difference). The HTTP server is listening only for
>>>>>>>> HTTPS
>>>>>>>> on 23270/tcp.
>>>>>>>> When I access these DMZ webservers via the F5 load balancer (to
>>>>>>>> which I
>>>>>>>> dont have access to, but the network folks configure for me), it
>>>>>>>> hangs.
>>>>>>>> Eventually returns: server1 https listen = 172.1.1.1:23270
>>>>>>>>
>>>>>>>> https://loadbalancer.domain.com:23270/SelfService
>>>>>>>> cant load.
>>>>>>>>
>>>>>>>> No idea why the URL is being re-written with the ":23270".
>>>>>>>> I added static content to the server.xml on 10.1.1.1 (Tomcat) to
>>>>>>>> test:
>>>>>>>> <Context docBase="/path/to/tomcat/static" path="/static" />
>>>>>>>> Then put a simple index.html in there. Accessing via the Apache Web
>>>>>>>> Servers works fine, but if you hit it with the Load Balancer it once
>>>>>>>> again
>>>>>>>> adds the https://loadbalancer.domain.com:23270/static
>>>>>>>>
>>>>>>>> Hitting https://loadbalancer.domain.com
>>>>>>>> I see my "Hello world!" which is all that is in index.html. This is
>>>>>>>> the
>>>>>>>> DocumentRoot of HTTP, and *not* proxied over at this time.
>>>>>>>>
>>>>>>>>
>>>>>>> So in this case, there is no delay, and you get the Apache
>>>>>>> httpd-hosted
>>>>>>> "index.html" containing "Hello World. Right ?
>>>>>>>
>>>>>>> Only
>>>>>>>
>>>>>>> /SelfService and /static are proxied
>>>>>>>> /static just being my test of static content, but still served up by
>>>>>>>> Tomcat..
>>>>>>>>
>>>>>>>> It's exactly 30 seconds before the page cannot be loaded when trying
>>>>>>>> anything proxied to Tomcat, but also accessed via the F5 load
>>>>>>>> balancer.
>>>>>>>> Not sure where the 30 seconds comes from; perhaps a load balancer
>>>>>>>> time
>>>>>>>> out,
>>>>>>>> as I dont see a "30" in my httpd configurations or my tomcat
>>>>>>>> server.xml
>>>>>>>>
>>>>>>>>
>>>>>>>> You can certainly look at the Apache httpd logs, and the tomcat
>>>>>>> logs, to
>>>>>>> see if you get a request or not.
>>>>>>> In Apache httpd, you can set the loglevel individually for mod_proxy
>>>>>>> (if
>>>>>>> you are running v 2.4), and it should show something if it gets this
>>>>>>> request and forwards it to tomcat.
>>>>>>> In tomcat, you can either enable an access log (which will show if it
>>>>>>> receives this request), or you could temporarily remove/rename the
>>>>>>> /static
>>>>>>> webapp. This way, it should trigger an error "not found" which you
>>>>>>> would
>>>>>>> also see in the error log.
>>>>>>>
>>>>>>> There should be nothing between them to hinder it. We have many load
>>>>>>>
>>>>>>>> balancers and this one specifically you dont need to open any
>>>>>>>> firewall
>>>>>>>> requests for the specific networks the HTTP servers are on. I did
>>>>>>>> have to
>>>>>>>> get the firewall opened up to allow me to hit
>>>>>>>> https://loadbalancer.domain.com because the VIP for "
>>>>>>>> loadbalancer.domain.com"
>>>>>>>> is in the DMZ, and my Desktop & VPN networks cannot hit it on 80/443
>>>>>>>> without opening holes. But beyond that, any connection from the F5
>>>>>>>> to the
>>>>>>>> HTTP Server should be 100% open bi-directional, since same subnet.
>>>>>>>>
>>>>>>>>
>>>>>>>> But something isn't working, otherwise you would not be asking.
>>>>>>> So,
>>>>>>>
>>>>>>> a) hitting the tomcat webapps through httpd seems to be working fine
>>>>>>> (browser -> httpd:23270 -> tomcat:18080 -> webapp or static)
>>>>>>>
>>>>>>> b) hitting a non-proxied-to-tomcat resource of httpd seems to work
>>>>>>> fine
>>>>>>> too, even through the F5
>>>>>>> (browser -> F5:443 -> httpd:23270 -> html page)
>>>>>>>
>>>>>>> c) it is only when you do :
>>>>>>> (browser -> F5:443 -> httpd:23270 -> tomcat:18080 -> webapp or
>>>>>>> static)
>>>>>>> that you see this issue
>>>>>>>
>>>>>>> It would really help if you looked in the logs of both httpd and
>>>>>>> tomcat,
>>>>>>> and checked for differences betweens cases a, b and c above.
>>>>>>>
>>>>>>> I believe that the F5 message with the port 23270 is a minor issue,
>>>>>>> of
>>>>>>> information disclosure by the F5, that it should not disclose.
>>>>>>>
>>>>>>> But the reason why it returns this error is obviously that in that
>>>>>>> case,
>>>>>>> it does not get a response from his request to httpd.
>>>>>>> The reason for this response not coming back to the F5 (in case c
>>>>>>> only),
>>>>>>> can be due to either httpd or tomcat. But F5 doesn't know about
>>>>>>> tomcat. So
>>>>>>> for the F5, it is httpd which is not responding. Thus,
>>>>>>> - either httpd is never getting the request from the F5 (unlikely,
>>>>>>> because
>>>>>>> in b above it gets it and responds)
>>>>>>> - or httpd is getting the request from the F5, but not forwarding it
>>>>>>> to
>>>>>>> tomcat, but also not returning an immediate error response to F5
>>>>>>> (which
>>>>>>> seems also unlikely, because of a and b)
>>>>>>> - or httpd is getting the request, forwarding it to tomcat, but not
>>>>>>> getting a response from tomcat. So
>>>>>>> - either tomcat is never getting the request from httpd (but in
>>>>>>> a, it
>>>>>>> gets it)
>>>>>>> - or tomcat is getting the request from httpd, but not responding
>>>>>>> (but
>>>>>>> in a, it does)
>>>>>>> - or tomcat is getting the request and responding, but the
>>>>>>> response
>>>>>>> never gets back to httpd (but in a, it does)
>>>>>>>
>>>>>>> So if a and b and c are all accurate, there is something apparently
>>>>>>> illogical happening.
>>>>>>> This would lead to the conclusion that a and b and c cannot all be
>>>>>>> accurate.
>>>>>>>
>>>>>>> The logs.. ?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Feb 21, 2017 at 2:05 PM, Antonio S. Cofino <
>>>>>>> cofinoa@gmail.com>
>>>>>>>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Aaron, on tomcat instances change the redirectPort attributte on
>>>>>>>> the http
>>>>>>>>
>>>>>>>>> conectó to the loabbalancer's port 443
>>>>>>>>>
>>>>>>>>> My guess is that your webapp has restriction rule requesting SSL
>>>>>>>>> con
>>>>>>>>> fidntial channel. Therefore the non-confidential to the 18080 port
>>>>>>>>> from
>>>>>>>>> the
>>>>>>>>> balancer are redirected to the 23270 port, but it should be 443.
>>>>>>>>>
>>>>>>>>> Antonio
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> El 21/2/2017 19:46, "Aaron Gray" <aa...@gmail.com> escribió:
>>>>>>>>>
>>>>>>>>> I have an application server from a vendor that comes bundled with
>>>>>>>>> an
>>>>>>>>> additional Apache Tomcat server. The webapp SelfService.war is
>>>>>>>>> vendor
>>>>>>>>> supplied too.
>>>>>>>>>
>>>>>>>>> Here's my problem (IP's replaced to protect the innocent):
>>>>>>>>>
>>>>>>>>> networks:
>>>>>>>>> DMZ=172.x.x.x
>>>>>>>>> INTERNAL=10.x.x.x
>>>>>>>>>
>>>>>>>>> server1 https listen = 172.1.1.1:23270
>>>>>>>>> server2 https listen = 172.1.1.2:23270
>>>>>>>>> F5 load balancer hostname = loadbalancer.domain.com:443
>>>>>>>>> backend tomcat server = 10.1.1.1:18080
>>>>>>>>>
>>>>>>>>> mod_proxy configuration:
>>>>>>>>> ProxyPass /SelfService http://10.1.1.1:18080/SelfService
>>>>>>>>> ProxyPassReverse /SelfService http://10.1.1.1:18080/SelfService
>>>>>>>>>
>>>>>>>>> When I access these DMZ webservers which mod_proxy back to Apache
>>>>>>>>> Tomcat
>>>>>>>>> as:
>>>>>>>>> https://172.1.1.1:23270/SelfService
>>>>>>>>> and
>>>>>>>>> https://172.1.1.2:23270/SelfService <https://172.1.1.1:23270/SelfS
>>>>>>>>> ervice
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> They load properly. Perfectly, every time!
>>>>>>>>>
>>>>>>>>> When I access these DMZ webservers via the F5 load balancer (to
>>>>>>>>> which I
>>>>>>>>> dont have access to, but the network folks configure for me), it
>>>>>>>>> hangs.
>>>>>>>>> Eventually returns:
>>>>>>>>> https://loadbalancer.domain.com:23270/SelfService
>>>>>>>>> cant load.
>>>>>>>>>
>>>>>>>>> No idea why the URL is being re-written with the ":23270".
>>>>>>>>> I added static content to the server.xml on 10.1.1.1 (Tomcat) to
>>>>>>>>> test:
>>>>>>>>> <Context docBase="/path/to/tomcat/static" path="/static" />
>>>>>>>>> Then put a simple index.html in there. Accessing via the Apache
>>>>>>>>> Web
>>>>>>>>> Servers works fine, but if you hit it with the Load Balancer it
>>>>>>>>> once
>>>>>>>>> again
>>>>>>>>> adds the https://loadbalancer.domain.com:23270/static
>>>>>>>>>
>>>>>>>>> Do you have any thoughts? Thanks so much, I have been working
>>>>>>>>> with this
>>>>>>>>> for weeks now with no success
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>> ------------------------------------------------------------
>>>>>>> ---------
>>>>>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>>>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>>>
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>>
>>>>
>>>
>>
>
Re: Strange URL rewrite when reverse proxy with Apache HTTP Server
Posted by Aaron Gray <aa...@gmail.com>.
Okay, guys, please accept my most sincere apology on this thing. I just
figured 1 thing out.
browser -> f5:443 -> httpd (23270, https) -> tomcat (http)
using the /static works! (see below)
If /static/index.html wasnt specified, then it hangs, then comes back as
https://loadbalancer.domain.com:23270/static
But if you go https://loadbalancer.domain.com/static/index.html it works!!!
DirectoryIndex index.html
is present. because if you just go to https://webserver1:23270/static
it works fine, no need for the extra /index.html
not sure why that is different there.
I only found this out using curl and just happened to think of it.
So you see the difference, from the tomcat access log, in the two
variations.
172.18.160.66 - - [23/Feb/2017:10:00:48 -0800] "GET /static HTTP/1.1" 302 -
172.18.160.66 - - [23/Feb/2017:10:03:04 -0800] "GET /static/index.html
HTTP/1.1" 200 32
Still /SelfService wont work though
I cant think of a way to do /SelfService/something...
Since it is a web app and i am no developer and its all from the vendor
(BMC Software) and it never shows naything other than /SelfService in the
URL. I dont think its a frame, but once again i dont know these things. I
know they told me it is HTML5, as it was using Silverlight back in the
older release.
On Thu, Feb 23, 2017 at 9:22 AM, Aaron Gray <aa...@gmail.com> wrote:
> Working with my F5 guy, we had an idea, since 80/tcp and 443/tcp were
> already open to the VIP on the F5, we simply turned of 80 -> 443 redirect
> on the F5, and then configured the F5 to use the non-HTTP port in Apache
> HTTP Server (the two backend servers). So its HTTP the entire way through,
> and the same issue persisted.
>
> browser -> F5(http) -> httpd(http port) -> tomcat(http port, nonssl)
>
> So i think i can rule out any possibility that the fact I was terminating
> SSL in Apache HTTP Server then proxying clear-text back as a reason. It
> worked fine like I said if you just go direct to the HTTP Server and slip
> the F5. And F5 works landing the /healthcheck.html (HTTP KeepAlive in the
> root, which isnt proxied), so all good there. Just cant /SelfService
> (configured w/ Proxy)
>
> thanks for your support here though.
>
> On Thu, Feb 23, 2017 at 9:12 AM, Aaron Gray <aa...@gmail.com> wrote:
>
>> SSLProxyEngine On
>> Was already turned on this entire time inside the ssl.conf (I include it)
>> VirtualHost section.
>>
>> I am debating turning on HTTPS in Tomcat on the backend 10.x.x.x app
>> server, and then HTTPS the whole way through and see if that makes any
>> difference. I may need to request a new firewall to be opened, which is
>> not able to open until 3/2. Gonna see.
>>
>>
>> On Thu, Feb 23, 2017 at 8:22 AM, Antonio S. Cofiño <co...@gmail.com>
>> wrote:
>>
>>>
>>> On 23/02/17 12:43, André Warnier (tomcat) wrote:
>>>
>>>> On 22.02.2017 19:22, Aaron Gray wrote:
>>>>
>>>>> So this is interesting.
>>>>>
>>>>> So from HTTP server #1 (172.1.1.1 example) I hit:
>>>>> https://172.1.1.1:23270/static and I see this in the HTTP log:
>>>>> 172.1.1.1 - - [22/Feb/2017:10:14:48 -0800] "GET /static/ HTTP/1.1" 200
>>>>> 32
>>>>> I see this in the Tomcat log:
>>>>> 172.1.1.1 - - [22/Feb/2017:10:14:48 -0800] "GET /static/ HTTP/1.1" 200
>>>>> 32
>>>>> ((( Yes I know they look identical, but the logformat is the same, so
>>>>> it
>>>>> comes out looking the same, but they are two diff log files totally
>>>>> )))
>>>>>
>>>>> I then hit the https://loadbalancer.domain.com/static from my Win10
>>>>> laptop
>>>>> (10.1.1.2 example)
>>>>> and I see this in the HTTP server #1 log, but NOTHING in the tomcat
>>>>> access
>>>>> log.
>>>>> 10.1.1.2 - - [22/Feb/2017:10:20:02 -0800] "-" 408 -
>>>>>
>>>>> So 408. Timeout. Hmm... Why? We KNOW that it can connect from http
>>>>> ->
>>>>> tomcat:18080 perfectly. So is it a timeout BACK to the F5? That i
>>>>> dont
>>>>> know yet.
>>>>>
>>>>
>>>> It looks as if, in that case, httpd is trying to connect to tomcat, but
>>>> - either it /is/ connecting, and sending the request, but tomcat never
>>>> answers
>>>> - or it is connecting to something else, which isn't tomcat
>>>> (but which accepts the request and doesn't answer either)
>>>>
>>>> I am not *sure*, but I believe, that if httpd was trying to connect to
>>>> something that isn't tomcat, and is not listening on port 18080, it would
>>>> get a different error, and a different logfile message. So it would seem
>>>> that at least the TCP connection works.
>>>>
>>>
>>>
>>> Is the F5 loadbalancer configured to make reverse proxy to a SSL http?
>>> The directive SSLProxyEngine onthe F5 loadbalencer has to be enabled.
>>> http://httpd.apache.org/docs/current/mod/mod_ssl.html#sslproxyengine
>>>
>>> If the loadbalancer it's delegating the SSL work to the server1
>>> server1 https listen = 172.1.1.1:23270
>>>
>>> then the F5 could be connecting as http regular TCP connection to a SSL
>>> TCP one and therefor the SSL handshake it's not happening.
>>>
>>> Antonio
>>>
>>>
>>>
>>>
>>>
>>>
>>>> Now there is still a puzzle there : when you connect as
>>>> browser --> httpd --> tomcat
>>>> then everything works : httpd connects to tomcat, sends a request, gets
>>>> a response, and returns it to the browser.
>>>>
>>>> But when you do the same as :
>>>> browser --> F5 --> httpd --> tomcat
>>>> it "does not work", and the problem seems to be at httpd --> tomat
>>>> Why would it make a difference in httpd --> tomcat, if it is either the
>>>> browser or the F5 which sends the request to httpd ?
>>>> the httpd --> tomcat connection should be the same, no matter what.
>>>>
>>>> Hmm, for now as puzzled as you are.
>>>> Maybe we have a quantum phenomenon here, where the mere fact of you
>>>> looking at the logfile, changes what happened beforehand. Just kidding.
>>>>
>>>> Informational note : the Apache access log line, is written when Apache
>>>> *has finished* processing the request (including the possible round-trip
>>>> through tomcat) and has sent the response to the client. (That's because,
>>>> for example, it can also write the time spent, and the size of the
>>>> response; so it has to know those before it can write the log line).
>>>> I presume that it is the same for tomcat.
>>>> Just mentioning this, so that you would maybe have a second look at the
>>>> tomcat access log, after a few minutes, to make sure that there really is
>>>> no corresponding line there.
>>>> It could be that tomcat actually /is/ receiving the request, and
>>>> processing it, but that it takes a long time, and that httpd tires of
>>>> waiting and already times out in the meantime. In such a case, you would
>>>> probably see an error some time later in the tomcat error log, when it is
>>>> trying to eventually return the response, and finds a closed connection
>>>> instead.
>>>> Now again, why it would do that in one case and not the other, is still
>>>> a mystery.
>>>>
>>>> Another note : in the "timeout" log line above, httpd is not showing
>>>> the request URL which triggered this timeout.
>>>> a) it may be showing some additional information in the Apache error log
>>>> b) these exists an httpd add-on module (mod_log_forensic ?) which could
>>>> provide more details in the log, about what happens precisely
>>>>
>>>>
>>>>
>>>>> On Tue, Feb 21, 2017 at 3:39 PM, André Warnier (tomcat) <aw@ice-sa.com
>>>>> >
>>>>> wrote:
>>>>>
>>>>> On 21.02.2017 23:28, Aaron Gray wrote:
>>>>>>
>>>>>> Antonio: The Tomcat server has no knowledge of the F5, or that it is
>>>>>>> being
>>>>>>> fronted by an Apache HTTP Server. I do SSL termination in Apache
>>>>>>> HTTP
>>>>>>> Server, and clear-text from HTTP to Tomcat.
>>>>>>> My redirect port for the normal HTTP listen in Tomcat is commented
>>>>>>> out.
>>>>>>> <Connector port="18080" protocol="HTTP/1.1"
>>>>>>> connectionTimeout="20000" />
>>>>>>> <!-- A "Connector" using the shared thread pool-->
>>>>>>> <!--
>>>>>>> <Connector executor="tomcatThreadPool"
>>>>>>> port="8080" protocol="HTTP/1.1"
>>>>>>> connectionTimeout="20000"
>>>>>>> redirectPort="8443" />
>>>>>>> -->
>>>>>>>
>>>>>>> Andre:
>>>>>>> The URL I am using is https://loadbalancer.domain.com
>>>>>>> It is listening on port 80 and 443, if you hit 80, internally it
>>>>>>> redirects
>>>>>>> you to 443. No SSL cert on the F5 load balancer. It simply sends
>>>>>>> the
>>>>>>> traffic to one of the two HTTP servers (round-robin, also tried
>>>>>>> persistence, no difference). The HTTP server is listening only for
>>>>>>> HTTPS
>>>>>>> on 23270/tcp.
>>>>>>> When I access these DMZ webservers via the F5 load balancer (to
>>>>>>> which I
>>>>>>> dont have access to, but the network folks configure for me), it
>>>>>>> hangs.
>>>>>>> Eventually returns: server1 https listen = 172.1.1.1:23270
>>>>>>>
>>>>>>> https://loadbalancer.domain.com:23270/SelfService
>>>>>>> cant load.
>>>>>>>
>>>>>>> No idea why the URL is being re-written with the ":23270".
>>>>>>> I added static content to the server.xml on 10.1.1.1 (Tomcat) to
>>>>>>> test:
>>>>>>> <Context docBase="/path/to/tomcat/static" path="/static" />
>>>>>>> Then put a simple index.html in there. Accessing via the Apache Web
>>>>>>> Servers works fine, but if you hit it with the Load Balancer it once
>>>>>>> again
>>>>>>> adds the https://loadbalancer.domain.com:23270/static
>>>>>>>
>>>>>>> Hitting https://loadbalancer.domain.com
>>>>>>> I see my "Hello world!" which is all that is in index.html. This is
>>>>>>> the
>>>>>>> DocumentRoot of HTTP, and *not* proxied over at this time.
>>>>>>>
>>>>>>>
>>>>>> So in this case, there is no delay, and you get the Apache
>>>>>> httpd-hosted
>>>>>> "index.html" containing "Hello World. Right ?
>>>>>>
>>>>>> Only
>>>>>>
>>>>>> /SelfService and /static are proxied
>>>>>>> /static just being my test of static content, but still served up by
>>>>>>> Tomcat..
>>>>>>>
>>>>>>> It's exactly 30 seconds before the page cannot be loaded when trying
>>>>>>> anything proxied to Tomcat, but also accessed via the F5 load
>>>>>>> balancer.
>>>>>>> Not sure where the 30 seconds comes from; perhaps a load balancer
>>>>>>> time
>>>>>>> out,
>>>>>>> as I dont see a "30" in my httpd configurations or my tomcat
>>>>>>> server.xml
>>>>>>>
>>>>>>>
>>>>>>> You can certainly look at the Apache httpd logs, and the tomcat
>>>>>> logs, to
>>>>>> see if you get a request or not.
>>>>>> In Apache httpd, you can set the loglevel individually for mod_proxy
>>>>>> (if
>>>>>> you are running v 2.4), and it should show something if it gets this
>>>>>> request and forwards it to tomcat.
>>>>>> In tomcat, you can either enable an access log (which will show if it
>>>>>> receives this request), or you could temporarily remove/rename the
>>>>>> /static
>>>>>> webapp. This way, it should trigger an error "not found" which you
>>>>>> would
>>>>>> also see in the error log.
>>>>>>
>>>>>> There should be nothing between them to hinder it. We have many load
>>>>>>
>>>>>>> balancers and this one specifically you dont need to open any
>>>>>>> firewall
>>>>>>> requests for the specific networks the HTTP servers are on. I did
>>>>>>> have to
>>>>>>> get the firewall opened up to allow me to hit
>>>>>>> https://loadbalancer.domain.com because the VIP for "
>>>>>>> loadbalancer.domain.com"
>>>>>>> is in the DMZ, and my Desktop & VPN networks cannot hit it on 80/443
>>>>>>> without opening holes. But beyond that, any connection from the F5
>>>>>>> to the
>>>>>>> HTTP Server should be 100% open bi-directional, since same subnet.
>>>>>>>
>>>>>>>
>>>>>>> But something isn't working, otherwise you would not be asking.
>>>>>> So,
>>>>>>
>>>>>> a) hitting the tomcat webapps through httpd seems to be working fine
>>>>>> (browser -> httpd:23270 -> tomcat:18080 -> webapp or static)
>>>>>>
>>>>>> b) hitting a non-proxied-to-tomcat resource of httpd seems to work
>>>>>> fine
>>>>>> too, even through the F5
>>>>>> (browser -> F5:443 -> httpd:23270 -> html page)
>>>>>>
>>>>>> c) it is only when you do :
>>>>>> (browser -> F5:443 -> httpd:23270 -> tomcat:18080 -> webapp or
>>>>>> static)
>>>>>> that you see this issue
>>>>>>
>>>>>> It would really help if you looked in the logs of both httpd and
>>>>>> tomcat,
>>>>>> and checked for differences betweens cases a, b and c above.
>>>>>>
>>>>>> I believe that the F5 message with the port 23270 is a minor issue, of
>>>>>> information disclosure by the F5, that it should not disclose.
>>>>>>
>>>>>> But the reason why it returns this error is obviously that in that
>>>>>> case,
>>>>>> it does not get a response from his request to httpd.
>>>>>> The reason for this response not coming back to the F5 (in case c
>>>>>> only),
>>>>>> can be due to either httpd or tomcat. But F5 doesn't know about
>>>>>> tomcat. So
>>>>>> for the F5, it is httpd which is not responding. Thus,
>>>>>> - either httpd is never getting the request from the F5 (unlikely,
>>>>>> because
>>>>>> in b above it gets it and responds)
>>>>>> - or httpd is getting the request from the F5, but not forwarding it
>>>>>> to
>>>>>> tomcat, but also not returning an immediate error response to F5
>>>>>> (which
>>>>>> seems also unlikely, because of a and b)
>>>>>> - or httpd is getting the request, forwarding it to tomcat, but not
>>>>>> getting a response from tomcat. So
>>>>>> - either tomcat is never getting the request from httpd (but in a,
>>>>>> it
>>>>>> gets it)
>>>>>> - or tomcat is getting the request from httpd, but not responding
>>>>>> (but
>>>>>> in a, it does)
>>>>>> - or tomcat is getting the request and responding, but the response
>>>>>> never gets back to httpd (but in a, it does)
>>>>>>
>>>>>> So if a and b and c are all accurate, there is something apparently
>>>>>> illogical happening.
>>>>>> This would lead to the conclusion that a and b and c cannot all be
>>>>>> accurate.
>>>>>>
>>>>>> The logs.. ?
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Tue, Feb 21, 2017 at 2:05 PM, Antonio S. Cofino <cofinoa@gmail.com
>>>>>> >
>>>>>>
>>>>>>> wrote:
>>>>>>>
>>>>>>> Aaron, on tomcat instances change the redirectPort attributte on the
>>>>>>> http
>>>>>>>
>>>>>>>> conectó to the loabbalancer's port 443
>>>>>>>>
>>>>>>>> My guess is that your webapp has restriction rule requesting SSL con
>>>>>>>> fidntial channel. Therefore the non-confidential to the 18080 port
>>>>>>>> from
>>>>>>>> the
>>>>>>>> balancer are redirected to the 23270 port, but it should be 443.
>>>>>>>>
>>>>>>>> Antonio
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> El 21/2/2017 19:46, "Aaron Gray" <aa...@gmail.com> escribió:
>>>>>>>>
>>>>>>>> I have an application server from a vendor that comes bundled with
>>>>>>>> an
>>>>>>>> additional Apache Tomcat server. The webapp SelfService.war is
>>>>>>>> vendor
>>>>>>>> supplied too.
>>>>>>>>
>>>>>>>> Here's my problem (IP's replaced to protect the innocent):
>>>>>>>>
>>>>>>>> networks:
>>>>>>>> DMZ=172.x.x.x
>>>>>>>> INTERNAL=10.x.x.x
>>>>>>>>
>>>>>>>> server1 https listen = 172.1.1.1:23270
>>>>>>>> server2 https listen = 172.1.1.2:23270
>>>>>>>> F5 load balancer hostname = loadbalancer.domain.com:443
>>>>>>>> backend tomcat server = 10.1.1.1:18080
>>>>>>>>
>>>>>>>> mod_proxy configuration:
>>>>>>>> ProxyPass /SelfService http://10.1.1.1:18080/SelfService
>>>>>>>> ProxyPassReverse /SelfService http://10.1.1.1:18080/SelfService
>>>>>>>>
>>>>>>>> When I access these DMZ webservers which mod_proxy back to Apache
>>>>>>>> Tomcat
>>>>>>>> as:
>>>>>>>> https://172.1.1.1:23270/SelfService
>>>>>>>> and
>>>>>>>> https://172.1.1.2:23270/SelfService <https://172.1.1.1:23270/SelfS
>>>>>>>> ervice
>>>>>>>>
>>>>>>>>>
>>>>>>>>> They load properly. Perfectly, every time!
>>>>>>>>
>>>>>>>> When I access these DMZ webservers via the F5 load balancer (to
>>>>>>>> which I
>>>>>>>> dont have access to, but the network folks configure for me), it
>>>>>>>> hangs.
>>>>>>>> Eventually returns:
>>>>>>>> https://loadbalancer.domain.com:23270/SelfService
>>>>>>>> cant load.
>>>>>>>>
>>>>>>>> No idea why the URL is being re-written with the ":23270".
>>>>>>>> I added static content to the server.xml on 10.1.1.1 (Tomcat) to
>>>>>>>> test:
>>>>>>>> <Context docBase="/path/to/tomcat/static" path="/static" />
>>>>>>>> Then put a simple index.html in there. Accessing via the Apache Web
>>>>>>>> Servers works fine, but if you hit it with the Load Balancer it once
>>>>>>>> again
>>>>>>>> adds the https://loadbalancer.domain.com:23270/static
>>>>>>>>
>>>>>>>> Do you have any thoughts? Thanks so much, I have been working with
>>>>>>>> this
>>>>>>>> for weeks now with no success
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>
>>>
>>
>
Re: Strange URL rewrite when reverse proxy with Apache HTTP Server
Posted by Aaron Gray <aa...@gmail.com>.
Working with my F5 guy, we had an idea, since 80/tcp and 443/tcp were
already open to the VIP on the F5, we simply turned of 80 -> 443 redirect
on the F5, and then configured the F5 to use the non-HTTP port in Apache
HTTP Server (the two backend servers). So its HTTP the entire way through,
and the same issue persisted.
browser -> F5(http) -> httpd(http port) -> tomcat(http port, nonssl)
So i think i can rule out any possibility that the fact I was terminating
SSL in Apache HTTP Server then proxying clear-text back as a reason. It
worked fine like I said if you just go direct to the HTTP Server and slip
the F5. And F5 works landing the /healthcheck.html (HTTP KeepAlive in the
root, which isnt proxied), so all good there. Just cant /SelfService
(configured w/ Proxy)
thanks for your support here though.
On Thu, Feb 23, 2017 at 9:12 AM, Aaron Gray <aa...@gmail.com> wrote:
> SSLProxyEngine On
> Was already turned on this entire time inside the ssl.conf (I include it)
> VirtualHost section.
>
> I am debating turning on HTTPS in Tomcat on the backend 10.x.x.x app
> server, and then HTTPS the whole way through and see if that makes any
> difference. I may need to request a new firewall to be opened, which is
> not able to open until 3/2. Gonna see.
>
>
> On Thu, Feb 23, 2017 at 8:22 AM, Antonio S. Cofiño <co...@gmail.com>
> wrote:
>
>>
>> On 23/02/17 12:43, André Warnier (tomcat) wrote:
>>
>>> On 22.02.2017 19:22, Aaron Gray wrote:
>>>
>>>> So this is interesting.
>>>>
>>>> So from HTTP server #1 (172.1.1.1 example) I hit:
>>>> https://172.1.1.1:23270/static and I see this in the HTTP log:
>>>> 172.1.1.1 - - [22/Feb/2017:10:14:48 -0800] "GET /static/ HTTP/1.1" 200
>>>> 32
>>>> I see this in the Tomcat log:
>>>> 172.1.1.1 - - [22/Feb/2017:10:14:48 -0800] "GET /static/ HTTP/1.1" 200
>>>> 32
>>>> ((( Yes I know they look identical, but the logformat is the same, so it
>>>> comes out looking the same, but they are two diff log files totally )))
>>>>
>>>> I then hit the https://loadbalancer.domain.com/static from my Win10
>>>> laptop
>>>> (10.1.1.2 example)
>>>> and I see this in the HTTP server #1 log, but NOTHING in the tomcat
>>>> access
>>>> log.
>>>> 10.1.1.2 - - [22/Feb/2017:10:20:02 -0800] "-" 408 -
>>>>
>>>> So 408. Timeout. Hmm... Why? We KNOW that it can connect from http ->
>>>> tomcat:18080 perfectly. So is it a timeout BACK to the F5? That i dont
>>>> know yet.
>>>>
>>>
>>> It looks as if, in that case, httpd is trying to connect to tomcat, but
>>> - either it /is/ connecting, and sending the request, but tomcat never
>>> answers
>>> - or it is connecting to something else, which isn't tomcat
>>> (but which accepts the request and doesn't answer either)
>>>
>>> I am not *sure*, but I believe, that if httpd was trying to connect to
>>> something that isn't tomcat, and is not listening on port 18080, it would
>>> get a different error, and a different logfile message. So it would seem
>>> that at least the TCP connection works.
>>>
>>
>>
>> Is the F5 loadbalancer configured to make reverse proxy to a SSL http?
>> The directive SSLProxyEngine onthe F5 loadbalencer has to be enabled.
>> http://httpd.apache.org/docs/current/mod/mod_ssl.html#sslproxyengine
>>
>> If the loadbalancer it's delegating the SSL work to the server1
>> server1 https listen = 172.1.1.1:23270
>>
>> then the F5 could be connecting as http regular TCP connection to a SSL
>> TCP one and therefor the SSL handshake it's not happening.
>>
>> Antonio
>>
>>
>>
>>
>>
>>
>>> Now there is still a puzzle there : when you connect as
>>> browser --> httpd --> tomcat
>>> then everything works : httpd connects to tomcat, sends a request, gets
>>> a response, and returns it to the browser.
>>>
>>> But when you do the same as :
>>> browser --> F5 --> httpd --> tomcat
>>> it "does not work", and the problem seems to be at httpd --> tomat
>>> Why would it make a difference in httpd --> tomcat, if it is either the
>>> browser or the F5 which sends the request to httpd ?
>>> the httpd --> tomcat connection should be the same, no matter what.
>>>
>>> Hmm, for now as puzzled as you are.
>>> Maybe we have a quantum phenomenon here, where the mere fact of you
>>> looking at the logfile, changes what happened beforehand. Just kidding.
>>>
>>> Informational note : the Apache access log line, is written when Apache
>>> *has finished* processing the request (including the possible round-trip
>>> through tomcat) and has sent the response to the client. (That's because,
>>> for example, it can also write the time spent, and the size of the
>>> response; so it has to know those before it can write the log line).
>>> I presume that it is the same for tomcat.
>>> Just mentioning this, so that you would maybe have a second look at the
>>> tomcat access log, after a few minutes, to make sure that there really is
>>> no corresponding line there.
>>> It could be that tomcat actually /is/ receiving the request, and
>>> processing it, but that it takes a long time, and that httpd tires of
>>> waiting and already times out in the meantime. In such a case, you would
>>> probably see an error some time later in the tomcat error log, when it is
>>> trying to eventually return the response, and finds a closed connection
>>> instead.
>>> Now again, why it would do that in one case and not the other, is still
>>> a mystery.
>>>
>>> Another note : in the "timeout" log line above, httpd is not showing the
>>> request URL which triggered this timeout.
>>> a) it may be showing some additional information in the Apache error log
>>> b) these exists an httpd add-on module (mod_log_forensic ?) which could
>>> provide more details in the log, about what happens precisely
>>>
>>>
>>>
>>>> On Tue, Feb 21, 2017 at 3:39 PM, André Warnier (tomcat) <aw...@ice-sa.com>
>>>> wrote:
>>>>
>>>> On 21.02.2017 23:28, Aaron Gray wrote:
>>>>>
>>>>> Antonio: The Tomcat server has no knowledge of the F5, or that it is
>>>>>> being
>>>>>> fronted by an Apache HTTP Server. I do SSL termination in Apache HTTP
>>>>>> Server, and clear-text from HTTP to Tomcat.
>>>>>> My redirect port for the normal HTTP listen in Tomcat is commented
>>>>>> out.
>>>>>> <Connector port="18080" protocol="HTTP/1.1"
>>>>>> connectionTimeout="20000" />
>>>>>> <!-- A "Connector" using the shared thread pool-->
>>>>>> <!--
>>>>>> <Connector executor="tomcatThreadPool"
>>>>>> port="8080" protocol="HTTP/1.1"
>>>>>> connectionTimeout="20000"
>>>>>> redirectPort="8443" />
>>>>>> -->
>>>>>>
>>>>>> Andre:
>>>>>> The URL I am using is https://loadbalancer.domain.com
>>>>>> It is listening on port 80 and 443, if you hit 80, internally it
>>>>>> redirects
>>>>>> you to 443. No SSL cert on the F5 load balancer. It simply sends the
>>>>>> traffic to one of the two HTTP servers (round-robin, also tried
>>>>>> persistence, no difference). The HTTP server is listening only for
>>>>>> HTTPS
>>>>>> on 23270/tcp.
>>>>>> When I access these DMZ webservers via the F5 load balancer (to which
>>>>>> I
>>>>>> dont have access to, but the network folks configure for me), it
>>>>>> hangs.
>>>>>> Eventually returns: server1 https listen = 172.1.1.1:23270
>>>>>>
>>>>>> https://loadbalancer.domain.com:23270/SelfService
>>>>>> cant load.
>>>>>>
>>>>>> No idea why the URL is being re-written with the ":23270".
>>>>>> I added static content to the server.xml on 10.1.1.1 (Tomcat) to test:
>>>>>> <Context docBase="/path/to/tomcat/static" path="/static" />
>>>>>> Then put a simple index.html in there. Accessing via the Apache Web
>>>>>> Servers works fine, but if you hit it with the Load Balancer it once
>>>>>> again
>>>>>> adds the https://loadbalancer.domain.com:23270/static
>>>>>>
>>>>>> Hitting https://loadbalancer.domain.com
>>>>>> I see my "Hello world!" which is all that is in index.html. This is
>>>>>> the
>>>>>> DocumentRoot of HTTP, and *not* proxied over at this time.
>>>>>>
>>>>>>
>>>>> So in this case, there is no delay, and you get the Apache httpd-hosted
>>>>> "index.html" containing "Hello World. Right ?
>>>>>
>>>>> Only
>>>>>
>>>>> /SelfService and /static are proxied
>>>>>> /static just being my test of static content, but still served up by
>>>>>> Tomcat..
>>>>>>
>>>>>> It's exactly 30 seconds before the page cannot be loaded when trying
>>>>>> anything proxied to Tomcat, but also accessed via the F5 load
>>>>>> balancer.
>>>>>> Not sure where the 30 seconds comes from; perhaps a load balancer time
>>>>>> out,
>>>>>> as I dont see a "30" in my httpd configurations or my tomcat
>>>>>> server.xml
>>>>>>
>>>>>>
>>>>>> You can certainly look at the Apache httpd logs, and the tomcat logs,
>>>>> to
>>>>> see if you get a request or not.
>>>>> In Apache httpd, you can set the loglevel individually for mod_proxy
>>>>> (if
>>>>> you are running v 2.4), and it should show something if it gets this
>>>>> request and forwards it to tomcat.
>>>>> In tomcat, you can either enable an access log (which will show if it
>>>>> receives this request), or you could temporarily remove/rename the
>>>>> /static
>>>>> webapp. This way, it should trigger an error "not found" which you
>>>>> would
>>>>> also see in the error log.
>>>>>
>>>>> There should be nothing between them to hinder it. We have many load
>>>>>
>>>>>> balancers and this one specifically you dont need to open any firewall
>>>>>> requests for the specific networks the HTTP servers are on. I did
>>>>>> have to
>>>>>> get the firewall opened up to allow me to hit
>>>>>> https://loadbalancer.domain.com because the VIP for "
>>>>>> loadbalancer.domain.com"
>>>>>> is in the DMZ, and my Desktop & VPN networks cannot hit it on 80/443
>>>>>> without opening holes. But beyond that, any connection from the F5
>>>>>> to the
>>>>>> HTTP Server should be 100% open bi-directional, since same subnet.
>>>>>>
>>>>>>
>>>>>> But something isn't working, otherwise you would not be asking.
>>>>> So,
>>>>>
>>>>> a) hitting the tomcat webapps through httpd seems to be working fine
>>>>> (browser -> httpd:23270 -> tomcat:18080 -> webapp or static)
>>>>>
>>>>> b) hitting a non-proxied-to-tomcat resource of httpd seems to work fine
>>>>> too, even through the F5
>>>>> (browser -> F5:443 -> httpd:23270 -> html page)
>>>>>
>>>>> c) it is only when you do :
>>>>> (browser -> F5:443 -> httpd:23270 -> tomcat:18080 -> webapp or
>>>>> static)
>>>>> that you see this issue
>>>>>
>>>>> It would really help if you looked in the logs of both httpd and
>>>>> tomcat,
>>>>> and checked for differences betweens cases a, b and c above.
>>>>>
>>>>> I believe that the F5 message with the port 23270 is a minor issue, of
>>>>> information disclosure by the F5, that it should not disclose.
>>>>>
>>>>> But the reason why it returns this error is obviously that in that
>>>>> case,
>>>>> it does not get a response from his request to httpd.
>>>>> The reason for this response not coming back to the F5 (in case c
>>>>> only),
>>>>> can be due to either httpd or tomcat. But F5 doesn't know about
>>>>> tomcat. So
>>>>> for the F5, it is httpd which is not responding. Thus,
>>>>> - either httpd is never getting the request from the F5 (unlikely,
>>>>> because
>>>>> in b above it gets it and responds)
>>>>> - or httpd is getting the request from the F5, but not forwarding it to
>>>>> tomcat, but also not returning an immediate error response to F5 (which
>>>>> seems also unlikely, because of a and b)
>>>>> - or httpd is getting the request, forwarding it to tomcat, but not
>>>>> getting a response from tomcat. So
>>>>> - either tomcat is never getting the request from httpd (but in a,
>>>>> it
>>>>> gets it)
>>>>> - or tomcat is getting the request from httpd, but not responding
>>>>> (but
>>>>> in a, it does)
>>>>> - or tomcat is getting the request and responding, but the response
>>>>> never gets back to httpd (but in a, it does)
>>>>>
>>>>> So if a and b and c are all accurate, there is something apparently
>>>>> illogical happening.
>>>>> This would lead to the conclusion that a and b and c cannot all be
>>>>> accurate.
>>>>>
>>>>> The logs.. ?
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Feb 21, 2017 at 2:05 PM, Antonio S. Cofino <co...@gmail.com>
>>>>>
>>>>>> wrote:
>>>>>>
>>>>>> Aaron, on tomcat instances change the redirectPort attributte on the
>>>>>> http
>>>>>>
>>>>>>> conectó to the loabbalancer's port 443
>>>>>>>
>>>>>>> My guess is that your webapp has restriction rule requesting SSL con
>>>>>>> fidntial channel. Therefore the non-confidential to the 18080 port
>>>>>>> from
>>>>>>> the
>>>>>>> balancer are redirected to the 23270 port, but it should be 443.
>>>>>>>
>>>>>>> Antonio
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> El 21/2/2017 19:46, "Aaron Gray" <aa...@gmail.com> escribió:
>>>>>>>
>>>>>>> I have an application server from a vendor that comes bundled with an
>>>>>>> additional Apache Tomcat server. The webapp SelfService.war is
>>>>>>> vendor
>>>>>>> supplied too.
>>>>>>>
>>>>>>> Here's my problem (IP's replaced to protect the innocent):
>>>>>>>
>>>>>>> networks:
>>>>>>> DMZ=172.x.x.x
>>>>>>> INTERNAL=10.x.x.x
>>>>>>>
>>>>>>> server1 https listen = 172.1.1.1:23270
>>>>>>> server2 https listen = 172.1.1.2:23270
>>>>>>> F5 load balancer hostname = loadbalancer.domain.com:443
>>>>>>> backend tomcat server = 10.1.1.1:18080
>>>>>>>
>>>>>>> mod_proxy configuration:
>>>>>>> ProxyPass /SelfService http://10.1.1.1:18080/SelfService
>>>>>>> ProxyPassReverse /SelfService http://10.1.1.1:18080/SelfService
>>>>>>>
>>>>>>> When I access these DMZ webservers which mod_proxy back to Apache
>>>>>>> Tomcat
>>>>>>> as:
>>>>>>> https://172.1.1.1:23270/SelfService
>>>>>>> and
>>>>>>> https://172.1.1.2:23270/SelfService <https://172.1.1.1:23270/SelfS
>>>>>>> ervice
>>>>>>>
>>>>>>>>
>>>>>>>> They load properly. Perfectly, every time!
>>>>>>>
>>>>>>> When I access these DMZ webservers via the F5 load balancer (to
>>>>>>> which I
>>>>>>> dont have access to, but the network folks configure for me), it
>>>>>>> hangs.
>>>>>>> Eventually returns:
>>>>>>> https://loadbalancer.domain.com:23270/SelfService
>>>>>>> cant load.
>>>>>>>
>>>>>>> No idea why the URL is being re-written with the ":23270".
>>>>>>> I added static content to the server.xml on 10.1.1.1 (Tomcat) to
>>>>>>> test:
>>>>>>> <Context docBase="/path/to/tomcat/static" path="/static" />
>>>>>>> Then put a simple index.html in there. Accessing via the Apache Web
>>>>>>> Servers works fine, but if you hit it with the Load Balancer it once
>>>>>>> again
>>>>>>> adds the https://loadbalancer.domain.com:23270/static
>>>>>>>
>>>>>>> Do you have any thoughts? Thanks so much, I have been working with
>>>>>>> this
>>>>>>> for weeks now with no success
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>>
>
Re: Strange URL rewrite when reverse proxy with Apache HTTP Server
Posted by Aaron Gray <aa...@gmail.com>.
SSLProxyEngine On
Was already turned on this entire time inside the ssl.conf (I include it)
VirtualHost section.
I am debating turning on HTTPS in Tomcat on the backend 10.x.x.x app
server, and then HTTPS the whole way through and see if that makes any
difference. I may need to request a new firewall to be opened, which is
not able to open until 3/2. Gonna see.
On Thu, Feb 23, 2017 at 8:22 AM, Antonio S. Cofiño <co...@gmail.com>
wrote:
>
> On 23/02/17 12:43, André Warnier (tomcat) wrote:
>
>> On 22.02.2017 19:22, Aaron Gray wrote:
>>
>>> So this is interesting.
>>>
>>> So from HTTP server #1 (172.1.1.1 example) I hit:
>>> https://172.1.1.1:23270/static and I see this in the HTTP log:
>>> 172.1.1.1 - - [22/Feb/2017:10:14:48 -0800] "GET /static/ HTTP/1.1" 200 32
>>> I see this in the Tomcat log:
>>> 172.1.1.1 - - [22/Feb/2017:10:14:48 -0800] "GET /static/ HTTP/1.1" 200 32
>>> ((( Yes I know they look identical, but the logformat is the same, so it
>>> comes out looking the same, but they are two diff log files totally )))
>>>
>>> I then hit the https://loadbalancer.domain.com/static from my Win10
>>> laptop
>>> (10.1.1.2 example)
>>> and I see this in the HTTP server #1 log, but NOTHING in the tomcat
>>> access
>>> log.
>>> 10.1.1.2 - - [22/Feb/2017:10:20:02 -0800] "-" 408 -
>>>
>>> So 408. Timeout. Hmm... Why? We KNOW that it can connect from http ->
>>> tomcat:18080 perfectly. So is it a timeout BACK to the F5? That i dont
>>> know yet.
>>>
>>
>> It looks as if, in that case, httpd is trying to connect to tomcat, but
>> - either it /is/ connecting, and sending the request, but tomcat never
>> answers
>> - or it is connecting to something else, which isn't tomcat
>> (but which accepts the request and doesn't answer either)
>>
>> I am not *sure*, but I believe, that if httpd was trying to connect to
>> something that isn't tomcat, and is not listening on port 18080, it would
>> get a different error, and a different logfile message. So it would seem
>> that at least the TCP connection works.
>>
>
>
> Is the F5 loadbalancer configured to make reverse proxy to a SSL http?
> The directive SSLProxyEngine onthe F5 loadbalencer has to be enabled.
> http://httpd.apache.org/docs/current/mod/mod_ssl.html#sslproxyengine
>
> If the loadbalancer it's delegating the SSL work to the server1
> server1 https listen = 172.1.1.1:23270
>
> then the F5 could be connecting as http regular TCP connection to a SSL
> TCP one and therefor the SSL handshake it's not happening.
>
> Antonio
>
>
>
>
>
>
>> Now there is still a puzzle there : when you connect as
>> browser --> httpd --> tomcat
>> then everything works : httpd connects to tomcat, sends a request, gets a
>> response, and returns it to the browser.
>>
>> But when you do the same as :
>> browser --> F5 --> httpd --> tomcat
>> it "does not work", and the problem seems to be at httpd --> tomat
>> Why would it make a difference in httpd --> tomcat, if it is either the
>> browser or the F5 which sends the request to httpd ?
>> the httpd --> tomcat connection should be the same, no matter what.
>>
>> Hmm, for now as puzzled as you are.
>> Maybe we have a quantum phenomenon here, where the mere fact of you
>> looking at the logfile, changes what happened beforehand. Just kidding.
>>
>> Informational note : the Apache access log line, is written when Apache
>> *has finished* processing the request (including the possible round-trip
>> through tomcat) and has sent the response to the client. (That's because,
>> for example, it can also write the time spent, and the size of the
>> response; so it has to know those before it can write the log line).
>> I presume that it is the same for tomcat.
>> Just mentioning this, so that you would maybe have a second look at the
>> tomcat access log, after a few minutes, to make sure that there really is
>> no corresponding line there.
>> It could be that tomcat actually /is/ receiving the request, and
>> processing it, but that it takes a long time, and that httpd tires of
>> waiting and already times out in the meantime. In such a case, you would
>> probably see an error some time later in the tomcat error log, when it is
>> trying to eventually return the response, and finds a closed connection
>> instead.
>> Now again, why it would do that in one case and not the other, is still a
>> mystery.
>>
>> Another note : in the "timeout" log line above, httpd is not showing the
>> request URL which triggered this timeout.
>> a) it may be showing some additional information in the Apache error log
>> b) these exists an httpd add-on module (mod_log_forensic ?) which could
>> provide more details in the log, about what happens precisely
>>
>>
>>
>>> On Tue, Feb 21, 2017 at 3:39 PM, André Warnier (tomcat) <aw...@ice-sa.com>
>>> wrote:
>>>
>>> On 21.02.2017 23:28, Aaron Gray wrote:
>>>>
>>>> Antonio: The Tomcat server has no knowledge of the F5, or that it is
>>>>> being
>>>>> fronted by an Apache HTTP Server. I do SSL termination in Apache HTTP
>>>>> Server, and clear-text from HTTP to Tomcat.
>>>>> My redirect port for the normal HTTP listen in Tomcat is commented out.
>>>>> <Connector port="18080" protocol="HTTP/1.1"
>>>>> connectionTimeout="20000" />
>>>>> <!-- A "Connector" using the shared thread pool-->
>>>>> <!--
>>>>> <Connector executor="tomcatThreadPool"
>>>>> port="8080" protocol="HTTP/1.1"
>>>>> connectionTimeout="20000"
>>>>> redirectPort="8443" />
>>>>> -->
>>>>>
>>>>> Andre:
>>>>> The URL I am using is https://loadbalancer.domain.com
>>>>> It is listening on port 80 and 443, if you hit 80, internally it
>>>>> redirects
>>>>> you to 443. No SSL cert on the F5 load balancer. It simply sends the
>>>>> traffic to one of the two HTTP servers (round-robin, also tried
>>>>> persistence, no difference). The HTTP server is listening only for
>>>>> HTTPS
>>>>> on 23270/tcp.
>>>>> When I access these DMZ webservers via the F5 load balancer (to which I
>>>>> dont have access to, but the network folks configure for me), it hangs.
>>>>> Eventually returns: server1 https listen = 172.1.1.1:23270
>>>>>
>>>>> https://loadbalancer.domain.com:23270/SelfService
>>>>> cant load.
>>>>>
>>>>> No idea why the URL is being re-written with the ":23270".
>>>>> I added static content to the server.xml on 10.1.1.1 (Tomcat) to test:
>>>>> <Context docBase="/path/to/tomcat/static" path="/static" />
>>>>> Then put a simple index.html in there. Accessing via the Apache Web
>>>>> Servers works fine, but if you hit it with the Load Balancer it once
>>>>> again
>>>>> adds the https://loadbalancer.domain.com:23270/static
>>>>>
>>>>> Hitting https://loadbalancer.domain.com
>>>>> I see my "Hello world!" which is all that is in index.html. This is the
>>>>> DocumentRoot of HTTP, and *not* proxied over at this time.
>>>>>
>>>>>
>>>> So in this case, there is no delay, and you get the Apache httpd-hosted
>>>> "index.html" containing "Hello World. Right ?
>>>>
>>>> Only
>>>>
>>>> /SelfService and /static are proxied
>>>>> /static just being my test of static content, but still served up by
>>>>> Tomcat..
>>>>>
>>>>> It's exactly 30 seconds before the page cannot be loaded when trying
>>>>> anything proxied to Tomcat, but also accessed via the F5 load balancer.
>>>>> Not sure where the 30 seconds comes from; perhaps a load balancer time
>>>>> out,
>>>>> as I dont see a "30" in my httpd configurations or my tomcat server.xml
>>>>>
>>>>>
>>>>> You can certainly look at the Apache httpd logs, and the tomcat logs,
>>>> to
>>>> see if you get a request or not.
>>>> In Apache httpd, you can set the loglevel individually for mod_proxy (if
>>>> you are running v 2.4), and it should show something if it gets this
>>>> request and forwards it to tomcat.
>>>> In tomcat, you can either enable an access log (which will show if it
>>>> receives this request), or you could temporarily remove/rename the
>>>> /static
>>>> webapp. This way, it should trigger an error "not found" which you would
>>>> also see in the error log.
>>>>
>>>> There should be nothing between them to hinder it. We have many load
>>>>
>>>>> balancers and this one specifically you dont need to open any firewall
>>>>> requests for the specific networks the HTTP servers are on. I did have
>>>>> to
>>>>> get the firewall opened up to allow me to hit
>>>>> https://loadbalancer.domain.com because the VIP for "
>>>>> loadbalancer.domain.com"
>>>>> is in the DMZ, and my Desktop & VPN networks cannot hit it on 80/443
>>>>> without opening holes. But beyond that, any connection from the F5 to
>>>>> the
>>>>> HTTP Server should be 100% open bi-directional, since same subnet.
>>>>>
>>>>>
>>>>> But something isn't working, otherwise you would not be asking.
>>>> So,
>>>>
>>>> a) hitting the tomcat webapps through httpd seems to be working fine
>>>> (browser -> httpd:23270 -> tomcat:18080 -> webapp or static)
>>>>
>>>> b) hitting a non-proxied-to-tomcat resource of httpd seems to work fine
>>>> too, even through the F5
>>>> (browser -> F5:443 -> httpd:23270 -> html page)
>>>>
>>>> c) it is only when you do :
>>>> (browser -> F5:443 -> httpd:23270 -> tomcat:18080 -> webapp or
>>>> static)
>>>> that you see this issue
>>>>
>>>> It would really help if you looked in the logs of both httpd and tomcat,
>>>> and checked for differences betweens cases a, b and c above.
>>>>
>>>> I believe that the F5 message with the port 23270 is a minor issue, of
>>>> information disclosure by the F5, that it should not disclose.
>>>>
>>>> But the reason why it returns this error is obviously that in that case,
>>>> it does not get a response from his request to httpd.
>>>> The reason for this response not coming back to the F5 (in case c only),
>>>> can be due to either httpd or tomcat. But F5 doesn't know about tomcat.
>>>> So
>>>> for the F5, it is httpd which is not responding. Thus,
>>>> - either httpd is never getting the request from the F5 (unlikely,
>>>> because
>>>> in b above it gets it and responds)
>>>> - or httpd is getting the request from the F5, but not forwarding it to
>>>> tomcat, but also not returning an immediate error response to F5 (which
>>>> seems also unlikely, because of a and b)
>>>> - or httpd is getting the request, forwarding it to tomcat, but not
>>>> getting a response from tomcat. So
>>>> - either tomcat is never getting the request from httpd (but in a, it
>>>> gets it)
>>>> - or tomcat is getting the request from httpd, but not responding
>>>> (but
>>>> in a, it does)
>>>> - or tomcat is getting the request and responding, but the response
>>>> never gets back to httpd (but in a, it does)
>>>>
>>>> So if a and b and c are all accurate, there is something apparently
>>>> illogical happening.
>>>> This would lead to the conclusion that a and b and c cannot all be
>>>> accurate.
>>>>
>>>> The logs.. ?
>>>>
>>>>
>>>>
>>>> On Tue, Feb 21, 2017 at 2:05 PM, Antonio S. Cofino <co...@gmail.com>
>>>>
>>>>> wrote:
>>>>>
>>>>> Aaron, on tomcat instances change the redirectPort attributte on the
>>>>> http
>>>>>
>>>>>> conectó to the loabbalancer's port 443
>>>>>>
>>>>>> My guess is that your webapp has restriction rule requesting SSL con
>>>>>> fidntial channel. Therefore the non-confidential to the 18080 port
>>>>>> from
>>>>>> the
>>>>>> balancer are redirected to the 23270 port, but it should be 443.
>>>>>>
>>>>>> Antonio
>>>>>>
>>>>>>
>>>>>>
>>>>>> El 21/2/2017 19:46, "Aaron Gray" <aa...@gmail.com> escribió:
>>>>>>
>>>>>> I have an application server from a vendor that comes bundled with an
>>>>>> additional Apache Tomcat server. The webapp SelfService.war is vendor
>>>>>> supplied too.
>>>>>>
>>>>>> Here's my problem (IP's replaced to protect the innocent):
>>>>>>
>>>>>> networks:
>>>>>> DMZ=172.x.x.x
>>>>>> INTERNAL=10.x.x.x
>>>>>>
>>>>>> server1 https listen = 172.1.1.1:23270
>>>>>> server2 https listen = 172.1.1.2:23270
>>>>>> F5 load balancer hostname = loadbalancer.domain.com:443
>>>>>> backend tomcat server = 10.1.1.1:18080
>>>>>>
>>>>>> mod_proxy configuration:
>>>>>> ProxyPass /SelfService http://10.1.1.1:18080/SelfService
>>>>>> ProxyPassReverse /SelfService http://10.1.1.1:18080/SelfService
>>>>>>
>>>>>> When I access these DMZ webservers which mod_proxy back to Apache
>>>>>> Tomcat
>>>>>> as:
>>>>>> https://172.1.1.1:23270/SelfService
>>>>>> and
>>>>>> https://172.1.1.2:23270/SelfService <https://172.1.1.1:23270/SelfS
>>>>>> ervice
>>>>>>
>>>>>>>
>>>>>>> They load properly. Perfectly, every time!
>>>>>>
>>>>>> When I access these DMZ webservers via the F5 load balancer (to which
>>>>>> I
>>>>>> dont have access to, but the network folks configure for me), it
>>>>>> hangs.
>>>>>> Eventually returns:
>>>>>> https://loadbalancer.domain.com:23270/SelfService
>>>>>> cant load.
>>>>>>
>>>>>> No idea why the URL is being re-written with the ":23270".
>>>>>> I added static content to the server.xml on 10.1.1.1 (Tomcat) to test:
>>>>>> <Context docBase="/path/to/tomcat/static" path="/static" />
>>>>>> Then put a simple index.html in there. Accessing via the Apache Web
>>>>>> Servers works fine, but if you hit it with the Load Balancer it once
>>>>>> again
>>>>>> adds the https://loadbalancer.domain.com:23270/static
>>>>>>
>>>>>> Do you have any thoughts? Thanks so much, I have been working with
>>>>>> this
>>>>>> for weeks now with no success
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>>
>>>>
>>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
Re: Strange URL rewrite when reverse proxy with Apache HTTP Server
Posted by "Antonio S. Cofiño" <co...@gmail.com>.
On 23/02/17 12:43, Andr� Warnier (tomcat) wrote:
> On 22.02.2017 19:22, Aaron Gray wrote:
>> So this is interesting.
>>
>> So from HTTP server #1 (172.1.1.1 example) I hit:
>> https://172.1.1.1:23270/static and I see this in the HTTP log:
>> 172.1.1.1 - - [22/Feb/2017:10:14:48 -0800] "GET /static/ HTTP/1.1"
>> 200 32
>> I see this in the Tomcat log:
>> 172.1.1.1 - - [22/Feb/2017:10:14:48 -0800] "GET /static/ HTTP/1.1"
>> 200 32
>> ((( Yes I know they look identical, but the logformat is the same, so it
>> comes out looking the same, but they are two diff log files totally )))
>>
>> I then hit the https://loadbalancer.domain.com/static from my Win10
>> laptop
>> (10.1.1.2 example)
>> and I see this in the HTTP server #1 log, but NOTHING in the tomcat
>> access
>> log.
>> 10.1.1.2 - - [22/Feb/2017:10:20:02 -0800] "-" 408 -
>>
>> So 408. Timeout. Hmm... Why? We KNOW that it can connect from http ->
>> tomcat:18080 perfectly. So is it a timeout BACK to the F5? That i dont
>> know yet.
>
> It looks as if, in that case, httpd is trying to connect to tomcat, but
> - either it /is/ connecting, and sending the request, but tomcat never
> answers
> - or it is connecting to something else, which isn't tomcat
> (but which accepts the request and doesn't answer either)
>
> I am not *sure*, but I believe, that if httpd was trying to connect to
> something that isn't tomcat, and is not listening on port 18080, it
> would get a different error, and a different logfile message. So it
> would seem that at least the TCP connection works.
Is the F5 loadbalancer configured to make reverse proxy to a SSL http?
The directive SSLProxyEngine onthe F5 loadbalencer has to be enabled.
http://httpd.apache.org/docs/current/mod/mod_ssl.html#sslproxyengine
If the loadbalancer it's delegating the SSL work to the server1
server1 https listen = 172.1.1.1:23270
then the F5 could be connecting as http regular TCP connection to a SSL
TCP one and therefor the SSL handshake it's not happening.
Antonio
>
> Now there is still a puzzle there : when you connect as
> browser --> httpd --> tomcat
> then everything works : httpd connects to tomcat, sends a request,
> gets a response, and returns it to the browser.
>
> But when you do the same as :
> browser --> F5 --> httpd --> tomcat
> it "does not work", and the problem seems to be at httpd --> tomat
> Why would it make a difference in httpd --> tomcat, if it is either
> the browser or the F5 which sends the request to httpd ?
> the httpd --> tomcat connection should be the same, no matter what.
>
> Hmm, for now as puzzled as you are.
> Maybe we have a quantum phenomenon here, where the mere fact of you
> looking at the logfile, changes what happened beforehand. Just kidding.
>
> Informational note : the Apache access log line, is written when
> Apache *has finished* processing the request (including the possible
> round-trip through tomcat) and has sent the response to the client.
> (That's because, for example, it can also write the time spent, and
> the size of the response; so it has to know those before it can write
> the log line).
> I presume that it is the same for tomcat.
> Just mentioning this, so that you would maybe have a second look at
> the tomcat access log, after a few minutes, to make sure that there
> really is no corresponding line there.
> It could be that tomcat actually /is/ receiving the request, and
> processing it, but that it takes a long time, and that httpd tires of
> waiting and already times out in the meantime. In such a case, you
> would probably see an error some time later in the tomcat error log,
> when it is trying to eventually return the response, and finds a
> closed connection instead.
> Now again, why it would do that in one case and not the other, is
> still a mystery.
>
> Another note : in the "timeout" log line above, httpd is not showing
> the request URL which triggered this timeout.
> a) it may be showing some additional information in the Apache error log
> b) these exists an httpd add-on module (mod_log_forensic ?) which
> could provide more details in the log, about what happens precisely
>
>
>>
>> On Tue, Feb 21, 2017 at 3:39 PM, Andr� Warnier (tomcat) <aw...@ice-sa.com>
>> wrote:
>>
>>> On 21.02.2017 23:28, Aaron Gray wrote:
>>>
>>>> Antonio: The Tomcat server has no knowledge of the F5, or that it is
>>>> being
>>>> fronted by an Apache HTTP Server. I do SSL termination in Apache HTTP
>>>> Server, and clear-text from HTTP to Tomcat.
>>>> My redirect port for the normal HTTP listen in Tomcat is commented
>>>> out.
>>>> <Connector port="18080" protocol="HTTP/1.1"
>>>> connectionTimeout="20000" />
>>>> <!-- A "Connector" using the shared thread pool-->
>>>> <!--
>>>> <Connector executor="tomcatThreadPool"
>>>> port="8080" protocol="HTTP/1.1"
>>>> connectionTimeout="20000"
>>>> redirectPort="8443" />
>>>> -->
>>>>
>>>> Andre:
>>>> The URL I am using is https://loadbalancer.domain.com
>>>> It is listening on port 80 and 443, if you hit 80, internally it
>>>> redirects
>>>> you to 443. No SSL cert on the F5 load balancer. It simply sends the
>>>> traffic to one of the two HTTP servers (round-robin, also tried
>>>> persistence, no difference). The HTTP server is listening only for
>>>> HTTPS
>>>> on 23270/tcp.
>>>> When I access these DMZ webservers via the F5 load balancer (to
>>>> which I
>>>> dont have access to, but the network folks configure for me), it
>>>> hangs.
>>>> Eventually returns: server1 https listen = 172.1.1.1:23270
>>>> https://loadbalancer.domain.com:23270/SelfService
>>>> cant load.
>>>>
>>>> No idea why the URL is being re-written with the ":23270".
>>>> I added static content to the server.xml on 10.1.1.1 (Tomcat) to test:
>>>> <Context docBase="/path/to/tomcat/static" path="/static" />
>>>> Then put a simple index.html in there. Accessing via the Apache Web
>>>> Servers works fine, but if you hit it with the Load Balancer it once
>>>> again
>>>> adds the https://loadbalancer.domain.com:23270/static
>>>>
>>>> Hitting https://loadbalancer.domain.com
>>>> I see my "Hello world!" which is all that is in index.html. This is
>>>> the
>>>> DocumentRoot of HTTP, and *not* proxied over at this time.
>>>>
>>>
>>> So in this case, there is no delay, and you get the Apache httpd-hosted
>>> "index.html" containing "Hello World. Right ?
>>>
>>> Only
>>>
>>>> /SelfService and /static are proxied
>>>> /static just being my test of static content, but still served up by
>>>> Tomcat..
>>>>
>>>> It's exactly 30 seconds before the page cannot be loaded when trying
>>>> anything proxied to Tomcat, but also accessed via the F5 load
>>>> balancer.
>>>> Not sure where the 30 seconds comes from; perhaps a load balancer time
>>>> out,
>>>> as I dont see a "30" in my httpd configurations or my tomcat
>>>> server.xml
>>>>
>>>>
>>> You can certainly look at the Apache httpd logs, and the tomcat
>>> logs, to
>>> see if you get a request or not.
>>> In Apache httpd, you can set the loglevel individually for mod_proxy
>>> (if
>>> you are running v 2.4), and it should show something if it gets this
>>> request and forwards it to tomcat.
>>> In tomcat, you can either enable an access log (which will show if it
>>> receives this request), or you could temporarily remove/rename the
>>> /static
>>> webapp. This way, it should trigger an error "not found" which you
>>> would
>>> also see in the error log.
>>>
>>> There should be nothing between them to hinder it. We have many load
>>>> balancers and this one specifically you dont need to open any firewall
>>>> requests for the specific networks the HTTP servers are on. I did
>>>> have to
>>>> get the firewall opened up to allow me to hit
>>>> https://loadbalancer.domain.com because the VIP for "
>>>> loadbalancer.domain.com"
>>>> is in the DMZ, and my Desktop & VPN networks cannot hit it on 80/443
>>>> without opening holes. But beyond that, any connection from the F5
>>>> to the
>>>> HTTP Server should be 100% open bi-directional, since same subnet.
>>>>
>>>>
>>> But something isn't working, otherwise you would not be asking.
>>> So,
>>>
>>> a) hitting the tomcat webapps through httpd seems to be working fine
>>> (browser -> httpd:23270 -> tomcat:18080 -> webapp or static)
>>>
>>> b) hitting a non-proxied-to-tomcat resource of httpd seems to work fine
>>> too, even through the F5
>>> (browser -> F5:443 -> httpd:23270 -> html page)
>>>
>>> c) it is only when you do :
>>> (browser -> F5:443 -> httpd:23270 -> tomcat:18080 -> webapp or
>>> static)
>>> that you see this issue
>>>
>>> It would really help if you looked in the logs of both httpd and
>>> tomcat,
>>> and checked for differences betweens cases a, b and c above.
>>>
>>> I believe that the F5 message with the port 23270 is a minor issue, of
>>> information disclosure by the F5, that it should not disclose.
>>>
>>> But the reason why it returns this error is obviously that in that
>>> case,
>>> it does not get a response from his request to httpd.
>>> The reason for this response not coming back to the F5 (in case c
>>> only),
>>> can be due to either httpd or tomcat. But F5 doesn't know about
>>> tomcat. So
>>> for the F5, it is httpd which is not responding. Thus,
>>> - either httpd is never getting the request from the F5 (unlikely,
>>> because
>>> in b above it gets it and responds)
>>> - or httpd is getting the request from the F5, but not forwarding it to
>>> tomcat, but also not returning an immediate error response to F5 (which
>>> seems also unlikely, because of a and b)
>>> - or httpd is getting the request, forwarding it to tomcat, but not
>>> getting a response from tomcat. So
>>> - either tomcat is never getting the request from httpd (but in
>>> a, it
>>> gets it)
>>> - or tomcat is getting the request from httpd, but not responding
>>> (but
>>> in a, it does)
>>> - or tomcat is getting the request and responding, but the response
>>> never gets back to httpd (but in a, it does)
>>>
>>> So if a and b and c are all accurate, there is something apparently
>>> illogical happening.
>>> This would lead to the conclusion that a and b and c cannot all be
>>> accurate.
>>>
>>> The logs.. ?
>>>
>>>
>>>
>>> On Tue, Feb 21, 2017 at 2:05 PM, Antonio S. Cofino <co...@gmail.com>
>>>> wrote:
>>>>
>>>> Aaron, on tomcat instances change the redirectPort attributte on
>>>> the http
>>>>> conect� to the loabbalancer's port 443
>>>>>
>>>>> My guess is that your webapp has restriction rule requesting SSL con
>>>>> fidntial channel. Therefore the non-confidential to the 18080 port
>>>>> from
>>>>> the
>>>>> balancer are redirected to the 23270 port, but it should be 443.
>>>>>
>>>>> Antonio
>>>>>
>>>>>
>>>>>
>>>>> El 21/2/2017 19:46, "Aaron Gray" <aa...@gmail.com> escribi�:
>>>>>
>>>>> I have an application server from a vendor that comes bundled with an
>>>>> additional Apache Tomcat server. The webapp SelfService.war is
>>>>> vendor
>>>>> supplied too.
>>>>>
>>>>> Here's my problem (IP's replaced to protect the innocent):
>>>>>
>>>>> networks:
>>>>> DMZ=172.x.x.x
>>>>> INTERNAL=10.x.x.x
>>>>>
>>>>> server1 https listen = 172.1.1.1:23270
>>>>> server2 https listen = 172.1.1.2:23270
>>>>> F5 load balancer hostname = loadbalancer.domain.com:443
>>>>> backend tomcat server = 10.1.1.1:18080
>>>>>
>>>>> mod_proxy configuration:
>>>>> ProxyPass /SelfService http://10.1.1.1:18080/SelfService
>>>>> ProxyPassReverse /SelfService http://10.1.1.1:18080/SelfService
>>>>>
>>>>> When I access these DMZ webservers which mod_proxy back to Apache
>>>>> Tomcat
>>>>> as:
>>>>> https://172.1.1.1:23270/SelfService
>>>>> and
>>>>> https://172.1.1.2:23270/SelfService
>>>>> <https://172.1.1.1:23270/SelfService
>>>>>>
>>>>> They load properly. Perfectly, every time!
>>>>>
>>>>> When I access these DMZ webservers via the F5 load balancer (to
>>>>> which I
>>>>> dont have access to, but the network folks configure for me), it
>>>>> hangs.
>>>>> Eventually returns:
>>>>> https://loadbalancer.domain.com:23270/SelfService
>>>>> cant load.
>>>>>
>>>>> No idea why the URL is being re-written with the ":23270".
>>>>> I added static content to the server.xml on 10.1.1.1 (Tomcat) to
>>>>> test:
>>>>> <Context docBase="/path/to/tomcat/static" path="/static" />
>>>>> Then put a simple index.html in there. Accessing via the Apache Web
>>>>> Servers works fine, but if you hit it with the Load Balancer it once
>>>>> again
>>>>> adds the https://loadbalancer.domain.com:23270/static
>>>>>
>>>>> Do you have any thoughts? Thanks so much, I have been working
>>>>> with this
>>>>> for weeks now with no success
>>>>>
>>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>
>>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Strange URL rewrite when reverse proxy with Apache HTTP Server
Posted by "André Warnier (tomcat)" <aw...@ice-sa.com>.
On 22.02.2017 19:22, Aaron Gray wrote:
> So this is interesting.
>
> So from HTTP server #1 (172.1.1.1 example) I hit:
> https://172.1.1.1:23270/static and I see this in the HTTP log:
> 172.1.1.1 - - [22/Feb/2017:10:14:48 -0800] "GET /static/ HTTP/1.1" 200 32
> I see this in the Tomcat log:
> 172.1.1.1 - - [22/Feb/2017:10:14:48 -0800] "GET /static/ HTTP/1.1" 200 32
> ((( Yes I know they look identical, but the logformat is the same, so it
> comes out looking the same, but they are two diff log files totally )))
>
> I then hit the https://loadbalancer.domain.com/static from my Win10 laptop
> (10.1.1.2 example)
> and I see this in the HTTP server #1 log, but NOTHING in the tomcat access
> log.
> 10.1.1.2 - - [22/Feb/2017:10:20:02 -0800] "-" 408 -
>
> So 408. Timeout. Hmm... Why? We KNOW that it can connect from http ->
> tomcat:18080 perfectly. So is it a timeout BACK to the F5? That i dont
> know yet.
It looks as if, in that case, httpd is trying to connect to tomcat, but
- either it /is/ connecting, and sending the request, but tomcat never answers
- or it is connecting to something else, which isn't tomcat
(but which accepts the request and doesn't answer either)
I am not *sure*, but I believe, that if httpd was trying to connect to something that
isn't tomcat, and is not listening on port 18080, it would get a different error, and a
different logfile message. So it would seem that at least the TCP connection works.
Now there is still a puzzle there : when you connect as
browser --> httpd --> tomcat
then everything works : httpd connects to tomcat, sends a request, gets a response, and
returns it to the browser.
But when you do the same as :
browser --> F5 --> httpd --> tomcat
it "does not work", and the problem seems to be at httpd --> tomat
Why would it make a difference in httpd --> tomcat, if it is either the browser or the F5
which sends the request to httpd ?
the httpd --> tomcat connection should be the same, no matter what.
Hmm, for now as puzzled as you are.
Maybe we have a quantum phenomenon here, where the mere fact of you looking at the
logfile, changes what happened beforehand. Just kidding.
Informational note : the Apache access log line, is written when Apache *has finished*
processing the request (including the possible round-trip through tomcat) and has sent the
response to the client. (That's because, for example, it can also write the time spent,
and the size of the response; so it has to know those before it can write the log line).
I presume that it is the same for tomcat.
Just mentioning this, so that you would maybe have a second look at the tomcat access log,
after a few minutes, to make sure that there really is no corresponding line there.
It could be that tomcat actually /is/ receiving the request, and processing it, but that
it takes a long time, and that httpd tires of waiting and already times out in the
meantime. In such a case, you would probably see an error some time later in the tomcat
error log, when it is trying to eventually return the response, and finds a closed
connection instead.
Now again, why it would do that in one case and not the other, is still a mystery.
Another note : in the "timeout" log line above, httpd is not showing the request URL which
triggered this timeout.
a) it may be showing some additional information in the Apache error log
b) these exists an httpd add-on module (mod_log_forensic ?) which could provide more
details in the log, about what happens precisely
>
> On Tue, Feb 21, 2017 at 3:39 PM, Andr� Warnier (tomcat) <aw...@ice-sa.com>
> wrote:
>
>> On 21.02.2017 23:28, Aaron Gray wrote:
>>
>>> Antonio: The Tomcat server has no knowledge of the F5, or that it is
>>> being
>>> fronted by an Apache HTTP Server. I do SSL termination in Apache HTTP
>>> Server, and clear-text from HTTP to Tomcat.
>>> My redirect port for the normal HTTP listen in Tomcat is commented out.
>>> <Connector port="18080" protocol="HTTP/1.1"
>>> connectionTimeout="20000" />
>>> <!-- A "Connector" using the shared thread pool-->
>>> <!--
>>> <Connector executor="tomcatThreadPool"
>>> port="8080" protocol="HTTP/1.1"
>>> connectionTimeout="20000"
>>> redirectPort="8443" />
>>> -->
>>>
>>> Andre:
>>> The URL I am using is https://loadbalancer.domain.com
>>> It is listening on port 80 and 443, if you hit 80, internally it redirects
>>> you to 443. No SSL cert on the F5 load balancer. It simply sends the
>>> traffic to one of the two HTTP servers (round-robin, also tried
>>> persistence, no difference). The HTTP server is listening only for HTTPS
>>> on 23270/tcp.
>>>
>>> Hitting https://loadbalancer.domain.com
>>> I see my "Hello world!" which is all that is in index.html. This is the
>>> DocumentRoot of HTTP, and *not* proxied over at this time.
>>>
>>
>> So in this case, there is no delay, and you get the Apache httpd-hosted
>> "index.html" containing "Hello World. Right ?
>>
>> Only
>>
>>> /SelfService and /static are proxied
>>> /static just being my test of static content, but still served up by
>>> Tomcat..
>>>
>>> It's exactly 30 seconds before the page cannot be loaded when trying
>>> anything proxied to Tomcat, but also accessed via the F5 load balancer.
>>> Not sure where the 30 seconds comes from; perhaps a load balancer time
>>> out,
>>> as I dont see a "30" in my httpd configurations or my tomcat server.xml
>>>
>>>
>> You can certainly look at the Apache httpd logs, and the tomcat logs, to
>> see if you get a request or not.
>> In Apache httpd, you can set the loglevel individually for mod_proxy (if
>> you are running v 2.4), and it should show something if it gets this
>> request and forwards it to tomcat.
>> In tomcat, you can either enable an access log (which will show if it
>> receives this request), or you could temporarily remove/rename the /static
>> webapp. This way, it should trigger an error "not found" which you would
>> also see in the error log.
>>
>> There should be nothing between them to hinder it. We have many load
>>> balancers and this one specifically you dont need to open any firewall
>>> requests for the specific networks the HTTP servers are on. I did have to
>>> get the firewall opened up to allow me to hit
>>> https://loadbalancer.domain.com because the VIP for "
>>> loadbalancer.domain.com"
>>> is in the DMZ, and my Desktop & VPN networks cannot hit it on 80/443
>>> without opening holes. But beyond that, any connection from the F5 to the
>>> HTTP Server should be 100% open bi-directional, since same subnet.
>>>
>>>
>> But something isn't working, otherwise you would not be asking.
>> So,
>>
>> a) hitting the tomcat webapps through httpd seems to be working fine
>> (browser -> httpd:23270 -> tomcat:18080 -> webapp or static)
>>
>> b) hitting a non-proxied-to-tomcat resource of httpd seems to work fine
>> too, even through the F5
>> (browser -> F5:443 -> httpd:23270 -> html page)
>>
>> c) it is only when you do :
>> (browser -> F5:443 -> httpd:23270 -> tomcat:18080 -> webapp or static)
>> that you see this issue
>>
>> It would really help if you looked in the logs of both httpd and tomcat,
>> and checked for differences betweens cases a, b and c above.
>>
>> I believe that the F5 message with the port 23270 is a minor issue, of
>> information disclosure by the F5, that it should not disclose.
>>
>> But the reason why it returns this error is obviously that in that case,
>> it does not get a response from his request to httpd.
>> The reason for this response not coming back to the F5 (in case c only),
>> can be due to either httpd or tomcat. But F5 doesn't know about tomcat. So
>> for the F5, it is httpd which is not responding. Thus,
>> - either httpd is never getting the request from the F5 (unlikely, because
>> in b above it gets it and responds)
>> - or httpd is getting the request from the F5, but not forwarding it to
>> tomcat, but also not returning an immediate error response to F5 (which
>> seems also unlikely, because of a and b)
>> - or httpd is getting the request, forwarding it to tomcat, but not
>> getting a response from tomcat. So
>> - either tomcat is never getting the request from httpd (but in a, it
>> gets it)
>> - or tomcat is getting the request from httpd, but not responding (but
>> in a, it does)
>> - or tomcat is getting the request and responding, but the response
>> never gets back to httpd (but in a, it does)
>>
>> So if a and b and c are all accurate, there is something apparently
>> illogical happening.
>> This would lead to the conclusion that a and b and c cannot all be
>> accurate.
>>
>> The logs.. ?
>>
>>
>>
>> On Tue, Feb 21, 2017 at 2:05 PM, Antonio S. Cofino <co...@gmail.com>
>>> wrote:
>>>
>>> Aaron, on tomcat instances change the redirectPort attributte on the http
>>>> conect� to the loabbalancer's port 443
>>>>
>>>> My guess is that your webapp has restriction rule requesting SSL con
>>>> fidntial channel. Therefore the non-confidential to the 18080 port from
>>>> the
>>>> balancer are redirected to the 23270 port, but it should be 443.
>>>>
>>>> Antonio
>>>>
>>>>
>>>>
>>>> El 21/2/2017 19:46, "Aaron Gray" <aa...@gmail.com> escribi�:
>>>>
>>>> I have an application server from a vendor that comes bundled with an
>>>> additional Apache Tomcat server. The webapp SelfService.war is vendor
>>>> supplied too.
>>>>
>>>> Here's my problem (IP's replaced to protect the innocent):
>>>>
>>>> networks:
>>>> DMZ=172.x.x.x
>>>> INTERNAL=10.x.x.x
>>>>
>>>> server1 https listen = 172.1.1.1:23270
>>>> server2 https listen = 172.1.1.2:23270
>>>> F5 load balancer hostname = loadbalancer.domain.com:443
>>>> backend tomcat server = 10.1.1.1:18080
>>>>
>>>> mod_proxy configuration:
>>>> ProxyPass /SelfService http://10.1.1.1:18080/SelfService
>>>> ProxyPassReverse /SelfService http://10.1.1.1:18080/SelfService
>>>>
>>>> When I access these DMZ webservers which mod_proxy back to Apache Tomcat
>>>> as:
>>>> https://172.1.1.1:23270/SelfService
>>>> and
>>>> https://172.1.1.2:23270/SelfService <https://172.1.1.1:23270/SelfService
>>>>>
>>>> They load properly. Perfectly, every time!
>>>>
>>>> When I access these DMZ webservers via the F5 load balancer (to which I
>>>> dont have access to, but the network folks configure for me), it hangs.
>>>> Eventually returns:
>>>> https://loadbalancer.domain.com:23270/SelfService
>>>> cant load.
>>>>
>>>> No idea why the URL is being re-written with the ":23270".
>>>> I added static content to the server.xml on 10.1.1.1 (Tomcat) to test:
>>>> <Context docBase="/path/to/tomcat/static" path="/static" />
>>>> Then put a simple index.html in there. Accessing via the Apache Web
>>>> Servers works fine, but if you hit it with the Load Balancer it once
>>>> again
>>>> adds the https://loadbalancer.domain.com:23270/static
>>>>
>>>> Do you have any thoughts? Thanks so much, I have been working with this
>>>> for weeks now with no success
>>>>
>>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Strange URL rewrite when reverse proxy with Apache HTTP Server
Posted by Aaron Gray <aa...@gmail.com>.
So this is interesting.
So from HTTP server #1 (172.1.1.1 example) I hit:
https://172.1.1.1:23270/static and I see this in the HTTP log:
172.1.1.1 - - [22/Feb/2017:10:14:48 -0800] "GET /static/ HTTP/1.1" 200 32
I see this in the Tomcat log:
172.1.1.1 - - [22/Feb/2017:10:14:48 -0800] "GET /static/ HTTP/1.1" 200 32
((( Yes I know they look identical, but the logformat is the same, so it
comes out looking the same, but they are two diff log files totally )))
I then hit the https://loadbalancer.domain.com/static from my Win10 laptop
(10.1.1.2 example)
and I see this in the HTTP server #1 log, but NOTHING in the tomcat access
log.
10.1.1.2 - - [22/Feb/2017:10:20:02 -0800] "-" 408 -
So 408. Timeout. Hmm... Why? We KNOW that it can connect from http ->
tomcat:18080 perfectly. So is it a timeout BACK to the F5? That i dont
know yet.
On Tue, Feb 21, 2017 at 3:39 PM, André Warnier (tomcat) <aw...@ice-sa.com>
wrote:
> On 21.02.2017 23:28, Aaron Gray wrote:
>
>> Antonio: The Tomcat server has no knowledge of the F5, or that it is
>> being
>> fronted by an Apache HTTP Server. I do SSL termination in Apache HTTP
>> Server, and clear-text from HTTP to Tomcat.
>> My redirect port for the normal HTTP listen in Tomcat is commented out.
>> <Connector port="18080" protocol="HTTP/1.1"
>> connectionTimeout="20000" />
>> <!-- A "Connector" using the shared thread pool-->
>> <!--
>> <Connector executor="tomcatThreadPool"
>> port="8080" protocol="HTTP/1.1"
>> connectionTimeout="20000"
>> redirectPort="8443" />
>> -->
>>
>> Andre:
>> The URL I am using is https://loadbalancer.domain.com
>> It is listening on port 80 and 443, if you hit 80, internally it redirects
>> you to 443. No SSL cert on the F5 load balancer. It simply sends the
>> traffic to one of the two HTTP servers (round-robin, also tried
>> persistence, no difference). The HTTP server is listening only for HTTPS
>> on 23270/tcp.
>>
>> Hitting https://loadbalancer.domain.com
>> I see my "Hello world!" which is all that is in index.html. This is the
>> DocumentRoot of HTTP, and *not* proxied over at this time.
>>
>
> So in this case, there is no delay, and you get the Apache httpd-hosted
> "index.html" containing "Hello World. Right ?
>
> Only
>
>> /SelfService and /static are proxied
>> /static just being my test of static content, but still served up by
>> Tomcat..
>>
>> It's exactly 30 seconds before the page cannot be loaded when trying
>> anything proxied to Tomcat, but also accessed via the F5 load balancer.
>> Not sure where the 30 seconds comes from; perhaps a load balancer time
>> out,
>> as I dont see a "30" in my httpd configurations or my tomcat server.xml
>>
>>
> You can certainly look at the Apache httpd logs, and the tomcat logs, to
> see if you get a request or not.
> In Apache httpd, you can set the loglevel individually for mod_proxy (if
> you are running v 2.4), and it should show something if it gets this
> request and forwards it to tomcat.
> In tomcat, you can either enable an access log (which will show if it
> receives this request), or you could temporarily remove/rename the /static
> webapp. This way, it should trigger an error "not found" which you would
> also see in the error log.
>
> There should be nothing between them to hinder it. We have many load
>> balancers and this one specifically you dont need to open any firewall
>> requests for the specific networks the HTTP servers are on. I did have to
>> get the firewall opened up to allow me to hit
>> https://loadbalancer.domain.com because the VIP for "
>> loadbalancer.domain.com"
>> is in the DMZ, and my Desktop & VPN networks cannot hit it on 80/443
>> without opening holes. But beyond that, any connection from the F5 to the
>> HTTP Server should be 100% open bi-directional, since same subnet.
>>
>>
> But something isn't working, otherwise you would not be asking.
> So,
>
> a) hitting the tomcat webapps through httpd seems to be working fine
> (browser -> httpd:23270 -> tomcat:18080 -> webapp or static)
>
> b) hitting a non-proxied-to-tomcat resource of httpd seems to work fine
> too, even through the F5
> (browser -> F5:443 -> httpd:23270 -> html page)
>
> c) it is only when you do :
> (browser -> F5:443 -> httpd:23270 -> tomcat:18080 -> webapp or static)
> that you see this issue
>
> It would really help if you looked in the logs of both httpd and tomcat,
> and checked for differences betweens cases a, b and c above.
>
> I believe that the F5 message with the port 23270 is a minor issue, of
> information disclosure by the F5, that it should not disclose.
>
> But the reason why it returns this error is obviously that in that case,
> it does not get a response from his request to httpd.
> The reason for this response not coming back to the F5 (in case c only),
> can be due to either httpd or tomcat. But F5 doesn't know about tomcat. So
> for the F5, it is httpd which is not responding. Thus,
> - either httpd is never getting the request from the F5 (unlikely, because
> in b above it gets it and responds)
> - or httpd is getting the request from the F5, but not forwarding it to
> tomcat, but also not returning an immediate error response to F5 (which
> seems also unlikely, because of a and b)
> - or httpd is getting the request, forwarding it to tomcat, but not
> getting a response from tomcat. So
> - either tomcat is never getting the request from httpd (but in a, it
> gets it)
> - or tomcat is getting the request from httpd, but not responding (but
> in a, it does)
> - or tomcat is getting the request and responding, but the response
> never gets back to httpd (but in a, it does)
>
> So if a and b and c are all accurate, there is something apparently
> illogical happening.
> This would lead to the conclusion that a and b and c cannot all be
> accurate.
>
> The logs.. ?
>
>
>
> On Tue, Feb 21, 2017 at 2:05 PM, Antonio S. Cofino <co...@gmail.com>
>> wrote:
>>
>> Aaron, on tomcat instances change the redirectPort attributte on the http
>>> conectó to the loabbalancer's port 443
>>>
>>> My guess is that your webapp has restriction rule requesting SSL con
>>> fidntial channel. Therefore the non-confidential to the 18080 port from
>>> the
>>> balancer are redirected to the 23270 port, but it should be 443.
>>>
>>> Antonio
>>>
>>>
>>>
>>> El 21/2/2017 19:46, "Aaron Gray" <aa...@gmail.com> escribió:
>>>
>>> I have an application server from a vendor that comes bundled with an
>>> additional Apache Tomcat server. The webapp SelfService.war is vendor
>>> supplied too.
>>>
>>> Here's my problem (IP's replaced to protect the innocent):
>>>
>>> networks:
>>> DMZ=172.x.x.x
>>> INTERNAL=10.x.x.x
>>>
>>> server1 https listen = 172.1.1.1:23270
>>> server2 https listen = 172.1.1.2:23270
>>> F5 load balancer hostname = loadbalancer.domain.com:443
>>> backend tomcat server = 10.1.1.1:18080
>>>
>>> mod_proxy configuration:
>>> ProxyPass /SelfService http://10.1.1.1:18080/SelfService
>>> ProxyPassReverse /SelfService http://10.1.1.1:18080/SelfService
>>>
>>> When I access these DMZ webservers which mod_proxy back to Apache Tomcat
>>> as:
>>> https://172.1.1.1:23270/SelfService
>>> and
>>> https://172.1.1.2:23270/SelfService <https://172.1.1.1:23270/SelfService
>>> >
>>> They load properly. Perfectly, every time!
>>>
>>> When I access these DMZ webservers via the F5 load balancer (to which I
>>> dont have access to, but the network folks configure for me), it hangs.
>>> Eventually returns:
>>> https://loadbalancer.domain.com:23270/SelfService
>>> cant load.
>>>
>>> No idea why the URL is being re-written with the ":23270".
>>> I added static content to the server.xml on 10.1.1.1 (Tomcat) to test:
>>> <Context docBase="/path/to/tomcat/static" path="/static" />
>>> Then put a simple index.html in there. Accessing via the Apache Web
>>> Servers works fine, but if you hit it with the Load Balancer it once
>>> again
>>> adds the https://loadbalancer.domain.com:23270/static
>>>
>>> Do you have any thoughts? Thanks so much, I have been working with this
>>> for weeks now with no success
>>>
>>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
Re: Strange URL rewrite when reverse proxy with Apache HTTP Server
Posted by Aaron Gray <aa...@gmail.com>.
Honestly, thank you for your replies and helping me step through this. I
was at my wits end with no known place to go. I appreciate it, truly.
I have to leave now to be somewhere in like 15 minutes, so I cant really
get in to all this now, but I read it. and I will dedicate time to it
tomorrow at work.
Yes:
A works
B works
C does not work
I am waiting on the vendor BMC Software to get back to me on this.
I should add also that I have another application which is not Tomcat, but
I do the same F5 -> Apache HTTP Server (2.2.31) -> App Server, which HTTPS
termination on Apache HTTP Server, and it works perfectly. I asked the
Network Folks to compare the configs and they say the config is good. I
will also escalate here at my company and perhaps when I have even more
ammo (logs per your request and such) to get F5 support involved.
I suspect they'll probably try to point the finger elsewhere, which will
make this an even bigger hassle.
On Tue, Feb 21, 2017 at 3:39 PM, André Warnier (tomcat) <aw...@ice-sa.com>
wrote:
> On 21.02.2017 23:28, Aaron Gray wrote:
>
>> Antonio: The Tomcat server has no knowledge of the F5, or that it is
>> being
>> fronted by an Apache HTTP Server. I do SSL termination in Apache HTTP
>> Server, and clear-text from HTTP to Tomcat.
>> My redirect port for the normal HTTP listen in Tomcat is commented out.
>> <Connector port="18080" protocol="HTTP/1.1"
>> connectionTimeout="20000" />
>> <!-- A "Connector" using the shared thread pool-->
>> <!--
>> <Connector executor="tomcatThreadPool"
>> port="8080" protocol="HTTP/1.1"
>> connectionTimeout="20000"
>> redirectPort="8443" />
>> -->
>>
>> Andre:
>> The URL I am using is https://loadbalancer.domain.com
>> It is listening on port 80 and 443, if you hit 80, internally it redirects
>> you to 443. No SSL cert on the F5 load balancer. It simply sends the
>> traffic to one of the two HTTP servers (round-robin, also tried
>> persistence, no difference). The HTTP server is listening only for HTTPS
>> on 23270/tcp.
>>
>> Hitting https://loadbalancer.domain.com
>> I see my "Hello world!" which is all that is in index.html. This is the
>> DocumentRoot of HTTP, and *not* proxied over at this time.
>>
>
> So in this case, there is no delay, and you get the Apache httpd-hosted
> "index.html" containing "Hello World. Right ?
>
> Only
>
>> /SelfService and /static are proxied
>> /static just being my test of static content, but still served up by
>> Tomcat..
>>
>> It's exactly 30 seconds before the page cannot be loaded when trying
>> anything proxied to Tomcat, but also accessed via the F5 load balancer.
>> Not sure where the 30 seconds comes from; perhaps a load balancer time
>> out,
>> as I dont see a "30" in my httpd configurations or my tomcat server.xml
>>
>>
> You can certainly look at the Apache httpd logs, and the tomcat logs, to
> see if you get a request or not.
> In Apache httpd, you can set the loglevel individually for mod_proxy (if
> you are running v 2.4), and it should show something if it gets this
> request and forwards it to tomcat.
> In tomcat, you can either enable an access log (which will show if it
> receives this request), or you could temporarily remove/rename the /static
> webapp. This way, it should trigger an error "not found" which you would
> also see in the error log.
>
> There should be nothing between them to hinder it. We have many load
>> balancers and this one specifically you dont need to open any firewall
>> requests for the specific networks the HTTP servers are on. I did have to
>> get the firewall opened up to allow me to hit
>> https://loadbalancer.domain.com because the VIP for "
>> loadbalancer.domain.com"
>> is in the DMZ, and my Desktop & VPN networks cannot hit it on 80/443
>> without opening holes. But beyond that, any connection from the F5 to the
>> HTTP Server should be 100% open bi-directional, since same subnet.
>>
>>
> But something isn't working, otherwise you would not be asking.
> So,
>
> a) hitting the tomcat webapps through httpd seems to be working fine
> (browser -> httpd:23270 -> tomcat:18080 -> webapp or static)
>
> b) hitting a non-proxied-to-tomcat resource of httpd seems to work fine
> too, even through the F5
> (browser -> F5:443 -> httpd:23270 -> html page)
>
> c) it is only when you do :
> (browser -> F5:443 -> httpd:23270 -> tomcat:18080 -> webapp or static)
> that you see this issue
>
> It would really help if you looked in the logs of both httpd and tomcat,
> and checked for differences betweens cases a, b and c above.
>
> I believe that the F5 message with the port 23270 is a minor issue, of
> information disclosure by the F5, that it should not disclose.
>
> But the reason why it returns this error is obviously that in that case,
> it does not get a response from his request to httpd.
> The reason for this response not coming back to the F5 (in case c only),
> can be due to either httpd or tomcat. But F5 doesn't know about tomcat. So
> for the F5, it is httpd which is not responding. Thus,
> - either httpd is never getting the request from the F5 (unlikely, because
> in b above it gets it and responds)
> - or httpd is getting the request from the F5, but not forwarding it to
> tomcat, but also not returning an immediate error response to F5 (which
> seems also unlikely, because of a and b)
> - or httpd is getting the request, forwarding it to tomcat, but not
> getting a response from tomcat. So
> - either tomcat is never getting the request from httpd (but in a, it
> gets it)
> - or tomcat is getting the request from httpd, but not responding (but
> in a, it does)
> - or tomcat is getting the request and responding, but the response
> never gets back to httpd (but in a, it does)
>
> So if a and b and c are all accurate, there is something apparently
> illogical happening.
> This would lead to the conclusion that a and b and c cannot all be
> accurate.
>
> The logs.. ?
>
>
>
> On Tue, Feb 21, 2017 at 2:05 PM, Antonio S. Cofino <co...@gmail.com>
>> wrote:
>>
>> Aaron, on tomcat instances change the redirectPort attributte on the http
>>> conectó to the loabbalancer's port 443
>>>
>>> My guess is that your webapp has restriction rule requesting SSL con
>>> fidntial channel. Therefore the non-confidential to the 18080 port from
>>> the
>>> balancer are redirected to the 23270 port, but it should be 443.
>>>
>>> Antonio
>>>
>>>
>>>
>>> El 21/2/2017 19:46, "Aaron Gray" <aa...@gmail.com> escribió:
>>>
>>> I have an application server from a vendor that comes bundled with an
>>> additional Apache Tomcat server. The webapp SelfService.war is vendor
>>> supplied too.
>>>
>>> Here's my problem (IP's replaced to protect the innocent):
>>>
>>> networks:
>>> DMZ=172.x.x.x
>>> INTERNAL=10.x.x.x
>>>
>>> server1 https listen = 172.1.1.1:23270
>>> server2 https listen = 172.1.1.2:23270
>>> F5 load balancer hostname = loadbalancer.domain.com:443
>>> backend tomcat server = 10.1.1.1:18080
>>>
>>> mod_proxy configuration:
>>> ProxyPass /SelfService http://10.1.1.1:18080/SelfService
>>> ProxyPassReverse /SelfService http://10.1.1.1:18080/SelfService
>>>
>>> When I access these DMZ webservers which mod_proxy back to Apache Tomcat
>>> as:
>>> https://172.1.1.1:23270/SelfService
>>> and
>>> https://172.1.1.2:23270/SelfService <https://172.1.1.1:23270/SelfService
>>> >
>>> They load properly. Perfectly, every time!
>>>
>>> When I access these DMZ webservers via the F5 load balancer (to which I
>>> dont have access to, but the network folks configure for me), it hangs.
>>> Eventually returns:
>>> https://loadbalancer.domain.com:23270/SelfService
>>> cant load.
>>>
>>> No idea why the URL is being re-written with the ":23270".
>>> I added static content to the server.xml on 10.1.1.1 (Tomcat) to test:
>>> <Context docBase="/path/to/tomcat/static" path="/static" />
>>> Then put a simple index.html in there. Accessing via the Apache Web
>>> Servers works fine, but if you hit it with the Load Balancer it once
>>> again
>>> adds the https://loadbalancer.domain.com:23270/static
>>>
>>> Do you have any thoughts? Thanks so much, I have been working with this
>>> for weeks now with no success
>>>
>>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
Re: Strange URL rewrite when reverse proxy with Apache HTTP Server
Posted by "André Warnier (tomcat)" <aw...@ice-sa.com>.
On 21.02.2017 23:28, Aaron Gray wrote:
> Antonio: The Tomcat server has no knowledge of the F5, or that it is being
> fronted by an Apache HTTP Server. I do SSL termination in Apache HTTP
> Server, and clear-text from HTTP to Tomcat.
> My redirect port for the normal HTTP listen in Tomcat is commented out.
> <Connector port="18080" protocol="HTTP/1.1"
> connectionTimeout="20000" />
> <!-- A "Connector" using the shared thread pool-->
> <!--
> <Connector executor="tomcatThreadPool"
> port="8080" protocol="HTTP/1.1"
> connectionTimeout="20000"
> redirectPort="8443" />
> -->
>
> Andre:
> The URL I am using is https://loadbalancer.domain.com
> It is listening on port 80 and 443, if you hit 80, internally it redirects
> you to 443. No SSL cert on the F5 load balancer. It simply sends the
> traffic to one of the two HTTP servers (round-robin, also tried
> persistence, no difference). The HTTP server is listening only for HTTPS
> on 23270/tcp.
>
> Hitting https://loadbalancer.domain.com
> I see my "Hello world!" which is all that is in index.html. This is the
> DocumentRoot of HTTP, and *not* proxied over at this time.
So in this case, there is no delay, and you get the Apache httpd-hosted "index.html"
containing "Hello World. Right ?
Only
> /SelfService and /static are proxied
> /static just being my test of static content, but still served up by Tomcat..
>
> It's exactly 30 seconds before the page cannot be loaded when trying
> anything proxied to Tomcat, but also accessed via the F5 load balancer.
> Not sure where the 30 seconds comes from; perhaps a load balancer time out,
> as I dont see a "30" in my httpd configurations or my tomcat server.xml
>
You can certainly look at the Apache httpd logs, and the tomcat logs, to see if you get a
request or not.
In Apache httpd, you can set the loglevel individually for mod_proxy (if you are running v
2.4), and it should show something if it gets this request and forwards it to tomcat.
In tomcat, you can either enable an access log (which will show if it receives this
request), or you could temporarily remove/rename the /static webapp. This way, it should
trigger an error "not found" which you would also see in the error log.
> There should be nothing between them to hinder it. We have many load
> balancers and this one specifically you dont need to open any firewall
> requests for the specific networks the HTTP servers are on. I did have to
> get the firewall opened up to allow me to hit
> https://loadbalancer.domain.com because the VIP for "loadbalancer.domain.com"
> is in the DMZ, and my Desktop & VPN networks cannot hit it on 80/443
> without opening holes. But beyond that, any connection from the F5 to the
> HTTP Server should be 100% open bi-directional, since same subnet.
>
But something isn't working, otherwise you would not be asking.
So,
a) hitting the tomcat webapps through httpd seems to be working fine
(browser -> httpd:23270 -> tomcat:18080 -> webapp or static)
b) hitting a non-proxied-to-tomcat resource of httpd seems to work fine too, even through
the F5
(browser -> F5:443 -> httpd:23270 -> html page)
c) it is only when you do :
(browser -> F5:443 -> httpd:23270 -> tomcat:18080 -> webapp or static)
that you see this issue
It would really help if you looked in the logs of both httpd and tomcat, and checked for
differences betweens cases a, b and c above.
I believe that the F5 message with the port 23270 is a minor issue, of information
disclosure by the F5, that it should not disclose.
But the reason why it returns this error is obviously that in that case, it does not get a
response from his request to httpd.
The reason for this response not coming back to the F5 (in case c only), can be due to
either httpd or tomcat. But F5 doesn't know about tomcat. So for the F5, it is httpd which
is not responding. Thus,
- either httpd is never getting the request from the F5 (unlikely, because in b above it
gets it and responds)
- or httpd is getting the request from the F5, but not forwarding it to tomcat, but also
not returning an immediate error response to F5 (which seems also unlikely, because of a
and b)
- or httpd is getting the request, forwarding it to tomcat, but not getting a response
from tomcat. So
- either tomcat is never getting the request from httpd (but in a, it gets it)
- or tomcat is getting the request from httpd, but not responding (but in a, it does)
- or tomcat is getting the request and responding, but the response never gets back to
httpd (but in a, it does)
So if a and b and c are all accurate, there is something apparently illogical happening.
This would lead to the conclusion that a and b and c cannot all be accurate.
The logs.. ?
> On Tue, Feb 21, 2017 at 2:05 PM, Antonio S. Cofino <co...@gmail.com>
> wrote:
>
>> Aaron, on tomcat instances change the redirectPort attributte on the http
>> conect� to the loabbalancer's port 443
>>
>> My guess is that your webapp has restriction rule requesting SSL con
>> fidntial channel. Therefore the non-confidential to the 18080 port from the
>> balancer are redirected to the 23270 port, but it should be 443.
>>
>> Antonio
>>
>>
>>
>> El 21/2/2017 19:46, "Aaron Gray" <aa...@gmail.com> escribi�:
>>
>> I have an application server from a vendor that comes bundled with an
>> additional Apache Tomcat server. The webapp SelfService.war is vendor
>> supplied too.
>>
>> Here's my problem (IP's replaced to protect the innocent):
>>
>> networks:
>> DMZ=172.x.x.x
>> INTERNAL=10.x.x.x
>>
>> server1 https listen = 172.1.1.1:23270
>> server2 https listen = 172.1.1.2:23270
>> F5 load balancer hostname = loadbalancer.domain.com:443
>> backend tomcat server = 10.1.1.1:18080
>>
>> mod_proxy configuration:
>> ProxyPass /SelfService http://10.1.1.1:18080/SelfService
>> ProxyPassReverse /SelfService http://10.1.1.1:18080/SelfService
>>
>> When I access these DMZ webservers which mod_proxy back to Apache Tomcat
>> as:
>> https://172.1.1.1:23270/SelfService
>> and
>> https://172.1.1.2:23270/SelfService <https://172.1.1.1:23270/SelfService>
>> They load properly. Perfectly, every time!
>>
>> When I access these DMZ webservers via the F5 load balancer (to which I
>> dont have access to, but the network folks configure for me), it hangs.
>> Eventually returns:
>> https://loadbalancer.domain.com:23270/SelfService
>> cant load.
>>
>> No idea why the URL is being re-written with the ":23270".
>> I added static content to the server.xml on 10.1.1.1 (Tomcat) to test:
>> <Context docBase="/path/to/tomcat/static" path="/static" />
>> Then put a simple index.html in there. Accessing via the Apache Web
>> Servers works fine, but if you hit it with the Load Balancer it once again
>> adds the https://loadbalancer.domain.com:23270/static
>>
>> Do you have any thoughts? Thanks so much, I have been working with this
>> for weeks now with no success
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Strange URL rewrite when reverse proxy with Apache HTTP Server
Posted by Aaron Gray <aa...@gmail.com>.
Antonio: The Tomcat server has no knowledge of the F5, or that it is being
fronted by an Apache HTTP Server. I do SSL termination in Apache HTTP
Server, and clear-text from HTTP to Tomcat.
My redirect port for the normal HTTP listen in Tomcat is commented out.
<Connector port="18080" protocol="HTTP/1.1"
connectionTimeout="20000" />
<!-- A "Connector" using the shared thread pool-->
<!--
<Connector executor="tomcatThreadPool"
port="8080" protocol="HTTP/1.1"
connectionTimeout="20000"
redirectPort="8443" />
-->
Andre:
The URL I am using is https://loadbalancer.domain.com
It is listening on port 80 and 443, if you hit 80, internally it redirects
you to 443. No SSL cert on the F5 load balancer. It simply sends the
traffic to one of the two HTTP servers (round-robin, also tried
persistence, no difference). The HTTP server is listening only for HTTPS
on 23270/tcp.
Hitting https://loadbalancer.domain.com
I see my "Hello world!" which is all that is in index.html. This is the
DocumentRoot of HTTP, and *not* proxied over at this time. Only
/SelfService and /static are proxied
/static just being my test of static content, but still served up by Tomcat.
It's exactly 30 seconds before the page cannot be loaded when trying
anything proxied to Tomcat, but also accessed via the F5 load balancer.
Not sure where the 30 seconds comes from; perhaps a load balancer time out,
as I dont see a "30" in my httpd configurations or my tomcat server.xml
There should be nothing between them to hinder it. We have many load
balancers and this one specifically you dont need to open any firewall
requests for the specific networks the HTTP servers are on. I did have to
get the firewall opened up to allow me to hit
https://loadbalancer.domain.com because the VIP for "loadbalancer.domain.com"
is in the DMZ, and my Desktop & VPN networks cannot hit it on 80/443
without opening holes. But beyond that, any connection from the F5 to the
HTTP Server should be 100% open bi-directional, since same subnet.
On Tue, Feb 21, 2017 at 2:05 PM, Antonio S. Cofino <co...@gmail.com>
wrote:
> Aaron, on tomcat instances change the redirectPort attributte on the http
> conectó to the loabbalancer's port 443
>
> My guess is that your webapp has restriction rule requesting SSL con
> fidntial channel. Therefore the non-confidential to the 18080 port from the
> balancer are redirected to the 23270 port, but it should be 443.
>
> Antonio
>
>
>
> El 21/2/2017 19:46, "Aaron Gray" <aa...@gmail.com> escribió:
>
> I have an application server from a vendor that comes bundled with an
> additional Apache Tomcat server. The webapp SelfService.war is vendor
> supplied too.
>
> Here's my problem (IP's replaced to protect the innocent):
>
> networks:
> DMZ=172.x.x.x
> INTERNAL=10.x.x.x
>
> server1 https listen = 172.1.1.1:23270
> server2 https listen = 172.1.1.2:23270
> F5 load balancer hostname = loadbalancer.domain.com:443
> backend tomcat server = 10.1.1.1:18080
>
> mod_proxy configuration:
> ProxyPass /SelfService http://10.1.1.1:18080/SelfService
> ProxyPassReverse /SelfService http://10.1.1.1:18080/SelfService
>
> When I access these DMZ webservers which mod_proxy back to Apache Tomcat
> as:
> https://172.1.1.1:23270/SelfService
> and
> https://172.1.1.2:23270/SelfService <https://172.1.1.1:23270/SelfService>
> They load properly. Perfectly, every time!
>
> When I access these DMZ webservers via the F5 load balancer (to which I
> dont have access to, but the network folks configure for me), it hangs.
> Eventually returns:
> https://loadbalancer.domain.com:23270/SelfService
> cant load.
>
> No idea why the URL is being re-written with the ":23270".
> I added static content to the server.xml on 10.1.1.1 (Tomcat) to test:
> <Context docBase="/path/to/tomcat/static" path="/static" />
> Then put a simple index.html in there. Accessing via the Apache Web
> Servers works fine, but if you hit it with the Load Balancer it once again
> adds the https://loadbalancer.domain.com:23270/static
>
> Do you have any thoughts? Thanks so much, I have been working with this
> for weeks now with no success
>
Re: Strange URL rewrite when reverse proxy with Apache HTTP Server
Posted by "Antonio S. Cofino" <co...@gmail.com>.
Aaron, on tomcat instances change the redirectPort attributte on the http
conectó to the loabbalancer's port 443
My guess is that your webapp has restriction rule requesting SSL con
fidntial channel. Therefore the non-confidential to the 18080 port from the
balancer are redirected to the 23270 port, but it should be 443.
Antonio
El 21/2/2017 19:46, "Aaron Gray" <aa...@gmail.com> escribió:
I have an application server from a vendor that comes bundled with an
additional Apache Tomcat server. The webapp SelfService.war is vendor
supplied too.
Here's my problem (IP's replaced to protect the innocent):
networks:
DMZ=172.x.x.x
INTERNAL=10.x.x.x
server1 https listen = 172.1.1.1:23270
server2 https listen = 172.1.1.2:23270
F5 load balancer hostname = loadbalancer.domain.com:443
backend tomcat server = 10.1.1.1:18080
mod_proxy configuration:
ProxyPass /SelfService http://10.1.1.1:18080/SelfService
ProxyPassReverse /SelfService http://10.1.1.1:18080/SelfService
When I access these DMZ webservers which mod_proxy back to Apache Tomcat as:
https://172.1.1.1:23270/SelfService
and
https://172.1.1.2:23270/SelfService <https://172.1.1.1:23270/SelfService>
They load properly. Perfectly, every time!
When I access these DMZ webservers via the F5 load balancer (to which I
dont have access to, but the network folks configure for me), it hangs.
Eventually returns:
https://loadbalancer.domain.com:23270/SelfService
cant load.
No idea why the URL is being re-written with the ":23270".
I added static content to the server.xml on 10.1.1.1 (Tomcat) to test:
<Context docBase="/path/to/tomcat/static" path="/static" />
Then put a simple index.html in there. Accessing via the Apache Web
Servers works fine, but if you hit it with the Load Balancer it once again
adds the https://loadbalancer.domain.com:23270/static
Do you have any thoughts? Thanks so much, I have been working with this
for weeks now with no success