You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Stefan Mayr <st...@mayr-stefan.de> on 2022/02/12 13:24:20 UTC
mod_jk interference with ErrorDocument/Alias on HEAD request
Hello Tomcat users,
this week we were debugging a strange connection issue which I tracked
down to an interference between Apache httpd and mod_jk.
For the full picture, the infrastructure setup contains
1. a Loadbalancer providing HTTPS, HTTP/1.1 and HTTP/2 for Clients.
2. an Apache httpd 2.4 webserver (HTTP only) with mod_jk
3. a Tomcat mit AJP-Connector
We have an application doing many different HEAD requests against an
application running in the Tomcat server. The requests contain an
Authorization header for Basic authentication. Expected response is a
HTTP 200 OK or HTTP 401 if this particular user is not allowed to access
that ressource. Because this is a HEAD request there must not be a
response body according to RFC 2616.
If there is a response body in the response to the HEAD request our
loadbalancer does strange things: aborts the connection if the clients
uses HTTP/2, SSL errors if using HTTP/1.1. But this is an issue on its
own which we might have to solve with the vendor.
Now comes the point where I need your help. We have a httpd
configuration with mod_jk which generates these invalid response bodies
on HEAD requests. I have a gut feeling this could be a bug with mod_jk.
For demonstration purpose i created a minimal demo app which only is a
WEB-INF/web.xml
<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns="https://jakarta.ee/xml/ns/jakartaee"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="https://jakarta.ee/xml/ns/jakartaee
https://jakarta.ee/xml/ns/jakartaee/web-app_5_0.xsd"
version="5.0">
<security-constraint>
<web-resource-collection>
<web-resource-name>Login</web-resource-name>
<url-pattern>/*</url-pattern>
</web-resource-collection>
<auth-constraint>
<role-name>manager</role-name>
</auth-constraint>
</security-constraint>
<security-role>
<role-name>manager</role-name>
</security-role>
<login-config>
<auth-method>BASIC</auth-method>
</login-config>
</web-app>
Then I place a JkMount in my Apache httpd configuration (+ minimal
worker.properties):
JkMount /demo/* ajp13_worker
Testing this with curl works like expected:
root@1ae8973f1b6b:~# curl -I -v localhost/demo/
* Trying 127.0.0.1:80...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 80 (#0)
> HEAD /demo/ HTTP/1.1
> Host: localhost
> User-Agent: curl/7.68.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 401 401
HTTP/1.1 401 401
< Date: Sat, 12 Feb 2022 12:57:33 GMT
Date: Sat, 12 Feb 2022 12:57:33 GMT
< Server: Apache/2.4.41 (Ubuntu)
Server: Apache/2.4.41 (Ubuntu)
< Cache-Control: private
Cache-Control: private
< WWW-Authenticate: Basic realm="Authentication required"
WWW-Authenticate: Basic realm="Authentication required"
< Content-Language: en
Content-Language: en
< Content-Type: text/html;charset=utf-8
Content-Type: text/html;charset=utf-8
<
* Connection #0 to host localhost left intact
But our default setup always includes custom error pages:
Alias /error/ "/usr/share/apache2/error/"
ErrorDocument 401 /error/HTTP_UNAUTHORIZED.html.var
If both of those lines are added this results in a response body for the
HEAD request.
root@1ae8973f1b6b:~# curl -I -v localhost/demo/
* Trying 127.0.0.1:80...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 80 (#0)
> HEAD /demo/ HTTP/1.1
> Host: localhost
> User-Agent: curl/7.68.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 401 401
HTTP/1.1 401 401
< Date: Sat, 12 Feb 2022 12:56:27 GMT
Date: Sat, 12 Feb 2022 12:56:27 GMT
< Server: Apache/2.4.41 (Ubuntu)
Server: Apache/2.4.41 (Ubuntu)
< Cache-Control: private
Cache-Control: private
< WWW-Authenticate: Basic realm="Authentication required"
WWW-Authenticate: Basic realm="Authentication required"
< Content-Language: en
Content-Language: en
< Content-Type: text/html;charset=utf-8
Content-Type: text/html;charset=utf-8
<
* Excess found: excess = 589 url = /demo/ (zero-length body)
* Connection #0 to host localhost left intact
Checking with tcpdump on port 8009 we see the expected response without
a body from the Tomcat AJP connector. The tcpdump von port 80 reveals
httpd is adding the configured ErrorDocument as response body.
If we comment out either the Alias or ErrorDocument directive the
response is correct again.
Doing similar tests with CGI/PHP applications always show the correct
response without a response body. This only affects requests which use
mod_jk.
So far I could reproduce this on SLES12 SP5 and SLES15 SP3 running
Apache httpd 2.4.51 and a self compile mod_jk 1.2.46 (same with the
included mod_jk 1.2.43) at work. At home the same happens on a stock
openSUSE Leap 15.3 with Apache httpd 2.4.51 and mod_jk 1.2.43 as well as
on Ubuntu 20.04 with httpd 2.4.41 and mod_jk 1.2.46.
I didn't try to compile the latest mod_jk version yet because I didn't
spot a relevant point in the changelog.
Can anyone confirm this behaviour or point me to a configuration
directive i missed?
Thank you,
Stefan Mayr
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: mod_jk interference with ErrorDocument/Alias on HEAD request
Posted by Stefan Mayr <st...@mayr-stefan.de>.
Hi Thomas,
Am 13.02.2022 um 17:27 schrieb Thomas Hoffmann (Speed4Trade GmbH):
> Hello,
>
> maybe you can try to set an environment variable which skips interpreting the content-length:
> https://tomcat.apache.org/connectors-doc/reference/apache.html#Advanced%20Environment%20Variables
> --> JK_IGNORE_CL
>
> To get more information, you can also set the logfile and log-level to debug: JkLogLevel
> (same reference page as above).
>
> Greetings,
> Thomas
I added both options:
Alias /error/ "/usr/share/apache2/error/"
ErrorDocument 401 /error/HTTP_UNAUTHORIZED.html.var
SetEnv JK_IGNORE_CL
JkLogLevel Debug
JkMount /demo/* ajp13_worker
This produces the following debug output:
[Sun Feb 13 16:37:02.591 2022] [212:140388647335680] [debug]
ajp_connection_tcp_get_message::jk_ajp_common.c (1459): 0050 01 00 17
74 65 78 74 2F 68 74 6D 6C 3B 63 68 61 - ...text/html;cha
[Sun Feb 13 16:37:02.591 2022] [212:140388647335680] [debug]
ajp_connection_tcp_get_message::jk_ajp_common.c (1459): 0060 72 73 65
74 3D 75 74 66 2D 38 00 A0 02 00 02 65 - rset=utf-8.....e
[Sun Feb 13 16:37:02.591 2022] [212:140388647335680] [debug]
ajp_connection_tcp_get_message::jk_ajp_common.c (1459): 0070 6E 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 - n...............
[Sun Feb 13 16:37:02.591 2022] [212:140388647335680] [debug]
ajp_unmarshal_response::jk_ajp_common.c (736): (ajp13_worker) status = 401
[Sun Feb 13 16:37:02.591 2022] [212:140388647335680] [debug]
ajp_unmarshal_response::jk_ajp_common.c (743): Number of headers is = 4
[Sun Feb 13 16:37:02.591 2022] [212:140388647335680] [debug]
ajp_unmarshal_response::jk_ajp_common.c (799): (ajp13_worker) Header[0]
[Cache-Control] = [private]
[Sun Feb 13 16:37:02.591 2022] [212:140388647335680] [debug]
ajp_unmarshal_response::jk_ajp_common.c (799): (ajp13_worker) Header[1]
[WWW-Authenticate] = [Basic realm="Authentication required"]
[Sun Feb 13 16:37:02.591 2022] [212:140388647335680] [debug]
ajp_unmarshal_response::jk_ajp_common.c (799): (ajp13_worker) Header[2]
[Content-Type] = [text/html;charset=utf-8]
[Sun Feb 13 16:37:02.591 2022] [212:140388647335680] [debug]
ajp_unmarshal_response::jk_ajp_common.c (799): (ajp13_worker) Header[3]
[Content-Language] = [en]
[Sun Feb 13 16:37:02.591 2022] [212:140388647335680] [debug]
ajp_connection_tcp_get_message::jk_ajp_common.c (1459): received from
ajp13 pos=0 len=2 max=8192
[Sun Feb 13 16:37:02.591 2022] [212:140388647335680] [debug]
ajp_connection_tcp_get_message::jk_ajp_common.c (1459): 0000 05 01 00
00 00 00 00 00 00 00 00 00 00 00 00 00 - ................
[Sun Feb 13 16:37:02.591 2022] [212:140388647335680] [debug]
ajp_process_callback::jk_ajp_common.c (2132): (ajp13_worker) AJP13
protocol: Reuse is OK
[Sun Feb 13 16:37:02.591 2022] [212:140388647335680] [debug]
ajp_reset_endpoint::jk_ajp_common.c (848): (ajp13_worker) resetting
endpoint with socket 15
[Sun Feb 13 16:37:02.591 2022] [212:140388647335680] [debug]
ajp_done::jk_ajp_common.c (3288): recycling connection pool for worker
ajp13_worker and socket 15
[Sun Feb 13 16:37:02.591 2022] [212:140388647335680] [info]
jk_handler::mod_jk.c (2968): No body with status=401 for worker=ajp13_worker
[Sun Feb 13 16:37:02.592 2022] [212:140388647335680] [debug]
jk_servlet_normalize::jk_util.c (2185): URI on entering
jk_servlet_normalize: [/error/HTTP_UNAUTHORIZED.html.var]
[Sun Feb 13 16:37:02.592 2022] [212:140388647335680] [debug]
jk_servlet_normalize::jk_util.c (2279): URI on exiting
jk_servlet_normalize: [/error/HTTP_UNAUTHORIZED.html.var]
[Sun Feb 13 16:37:02.592 2022] [212:140388647335680] [debug]
map_uri_to_worker_ext::jk_uri_worker_map.c (1168): Attempting to map URI
'/error/HTTP_UNAUTHORIZED.html.var' from 1 maps
[Sun Feb 13 16:37:02.592 2022] [212:140388647335680] [debug]
find_match::jk_uri_worker_map.c (978): Attempting to map context URI
'/demo/*=ajp13_worker' source 'JkMount'
[Sun Feb 13 16:37:02.592 2022] [212:140388647335680] [debug]
jk_map_to_storage::mod_jk.c (4058): no match for
/error/HTTP_UNAUTHORIZED.html.var found
The last lines are irritating. They don't change when I try to add a
JkUnMount:
JkMount /demo/* ajp13_worker
JkUnMount /error/* ajp13_worker
> -----Ursprüngliche Nachricht-----
> Von: Stefan Mayr <st...@mayr-stefan.de>
> Gesendet: Samstag, 12. Februar 2022 14:24
> An: Tomcat Users List <us...@tomcat.apache.org>
> Betreff: mod_jk interference with ErrorDocument/Alias on HEAD request
>
> Hello Tomcat users,
>
> this week we were debugging a strange connection issue which I tracked down to an interference between Apache httpd and mod_jk.
>
> For the full picture, the infrastructure setup contains
>
> 1. a Loadbalancer providing HTTPS, HTTP/1.1 and HTTP/2 for Clients.
> 2. an Apache httpd 2.4 webserver (HTTP only) with mod_jk 3. a Tomcat mit AJP-Connector
>
> We have an application doing many different HEAD requests against an application running in the Tomcat server. The requests contain an Authorization header for Basic authentication. Expected response is a HTTP 200 OK or HTTP 401 if this particular user is not allowed to access that ressource. Because this is a HEAD request there must not be a response body according to RFC 2616.
>
> If there is a response body in the response to the HEAD request our loadbalancer does strange things: aborts the connection if the clients uses HTTP/2, SSL errors if using HTTP/1.1. But this is an issue on its own which we might have to solve with the vendor.
>
> Now comes the point where I need your help. We have a httpd configuration with mod_jk which generates these invalid response bodies on HEAD requests. I have a gut feeling this could be a bug with mod_jk.
>
> For demonstration purpose i created a minimal demo app which only is a WEB-INF/web.xml <?xml version="1.0" encoding="UTF-8"?> <web-app xmlns="https://jakarta.ee/xml/ns/jakartaee"
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> xsi:schemaLocation="https://jakarta.ee/xml/ns/jakartaee
> https://jakarta.ee/xml/ns/jakartaee/web-app_5_0.xsd"
> version="5.0">
> <security-constraint>
> <web-resource-collection>
> <web-resource-name>Login</web-resource-name>
> <url-pattern>/*</url-pattern>
> </web-resource-collection>
> <auth-constraint>
> <role-name>manager</role-name>
> </auth-constraint>
> </security-constraint>
> <security-role>
> <role-name>manager</role-name>
> </security-role>
> <login-config>
> <auth-method>BASIC</auth-method>
> </login-config>
> </web-app>
>
> Then I place a JkMount in my Apache httpd configuration (+ minimal
> worker.properties):
>
> JkMount /demo/* ajp13_worker
>
> Testing this with curl works like expected:
>
> root@1ae8973f1b6b:~# curl -I -v localhost/demo/
> * Trying 127.0.0.1:80...
> * TCP_NODELAY set
> * Connected to localhost (127.0.0.1) port 80 (#0) > HEAD /demo/ HTTP/1.1 > Host: localhost > User-Agent: curl/7.68.0 > Accept: */* >
> * Mark bundle as not supporting multiuse < HTTP/1.1 401 401
> HTTP/1.1 401 401
> < Date: Sat, 12 Feb 2022 12:57:33 GMT
> Date: Sat, 12 Feb 2022 12:57:33 GMT
> < Server: Apache/2.4.41 (Ubuntu)
> Server: Apache/2.4.41 (Ubuntu)
> < Cache-Control: private
> Cache-Control: private
> < WWW-Authenticate: Basic realm="Authentication required"
> WWW-Authenticate: Basic realm="Authentication required"
> < Content-Language: en
> Content-Language: en
> < Content-Type: text/html;charset=utf-8
> Content-Type: text/html;charset=utf-8
>
> <
> * Connection #0 to host localhost left intact
>
> But our default setup always includes custom error pages:
>
> Alias /error/ "/usr/share/apache2/error/"
> ErrorDocument 401 /error/HTTP_UNAUTHORIZED.html.var
>
> If both of those lines are added this results in a response body for the HEAD request.
>
> root@1ae8973f1b6b:~# curl -I -v localhost/demo/
> * Trying 127.0.0.1:80...
> * TCP_NODELAY set
> * Connected to localhost (127.0.0.1) port 80 (#0) > HEAD /demo/ HTTP/1.1 > Host: localhost > User-Agent: curl/7.68.0 > Accept: */* >
> * Mark bundle as not supporting multiuse < HTTP/1.1 401 401
> HTTP/1.1 401 401
> < Date: Sat, 12 Feb 2022 12:56:27 GMT
> Date: Sat, 12 Feb 2022 12:56:27 GMT
> < Server: Apache/2.4.41 (Ubuntu)
> Server: Apache/2.4.41 (Ubuntu)
> < Cache-Control: private
> Cache-Control: private
> < WWW-Authenticate: Basic realm="Authentication required"
> WWW-Authenticate: Basic realm="Authentication required"
> < Content-Language: en
> Content-Language: en
> < Content-Type: text/html;charset=utf-8
> Content-Type: text/html;charset=utf-8
>
> <
> * Excess found: excess = 589 url = /demo/ (zero-length body)
> * Connection #0 to host localhost left intact
>
> Checking with tcpdump on port 8009 we see the expected response without a body from the Tomcat AJP connector. The tcpdump von port 80 reveals httpd is adding the configured ErrorDocument as response body.
>
> If we comment out either the Alias or ErrorDocument directive the response is correct again.
>
> Doing similar tests with CGI/PHP applications always show the correct response without a response body. This only affects requests which use mod_jk.
>
> So far I could reproduce this on SLES12 SP5 and SLES15 SP3 running Apache httpd 2.4.51 and a self compile mod_jk 1.2.46 (same with the included mod_jk 1.2.43) at work. At home the same happens on a stock openSUSE Leap 15.3 with Apache httpd 2.4.51 and mod_jk 1.2.43 as well as on Ubuntu 20.04 with httpd 2.4.41 and mod_jk 1.2.46.
> I didn't try to compile the latest mod_jk version yet because I didn't spot a relevant point in the changelog.
>
> Can anyone confirm this behaviour or point me to a configuration directive i missed?
>
> Thank you,
>
>
> Stefan Mayr
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
Regards,
Stefan Mayr
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
AW: mod_jk interference with ErrorDocument/Alias on HEAD request
Posted by "Thomas Hoffmann (Speed4Trade GmbH)" <Th...@speed4trade.com.INVALID>.
Hello,
maybe you can try to set an environment variable which skips interpreting the content-length:
https://tomcat.apache.org/connectors-doc/reference/apache.html#Advanced%20Environment%20Variables
--> JK_IGNORE_CL
To get more information, you can also set the logfile and log-level to debug: JkLogLevel
(same reference page as above).
Greetings,
Thomas
-----Ursprüngliche Nachricht-----
Von: Stefan Mayr <st...@mayr-stefan.de>
Gesendet: Samstag, 12. Februar 2022 14:24
An: Tomcat Users List <us...@tomcat.apache.org>
Betreff: mod_jk interference with ErrorDocument/Alias on HEAD request
Hello Tomcat users,
this week we were debugging a strange connection issue which I tracked down to an interference between Apache httpd and mod_jk.
For the full picture, the infrastructure setup contains
1. a Loadbalancer providing HTTPS, HTTP/1.1 and HTTP/2 for Clients.
2. an Apache httpd 2.4 webserver (HTTP only) with mod_jk 3. a Tomcat mit AJP-Connector
We have an application doing many different HEAD requests against an application running in the Tomcat server. The requests contain an Authorization header for Basic authentication. Expected response is a HTTP 200 OK or HTTP 401 if this particular user is not allowed to access that ressource. Because this is a HEAD request there must not be a response body according to RFC 2616.
If there is a response body in the response to the HEAD request our loadbalancer does strange things: aborts the connection if the clients uses HTTP/2, SSL errors if using HTTP/1.1. But this is an issue on its own which we might have to solve with the vendor.
Now comes the point where I need your help. We have a httpd configuration with mod_jk which generates these invalid response bodies on HEAD requests. I have a gut feeling this could be a bug with mod_jk.
For demonstration purpose i created a minimal demo app which only is a WEB-INF/web.xml <?xml version="1.0" encoding="UTF-8"?> <web-app xmlns="https://jakarta.ee/xml/ns/jakartaee"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="https://jakarta.ee/xml/ns/jakartaee
https://jakarta.ee/xml/ns/jakartaee/web-app_5_0.xsd"
version="5.0">
<security-constraint>
<web-resource-collection>
<web-resource-name>Login</web-resource-name>
<url-pattern>/*</url-pattern>
</web-resource-collection>
<auth-constraint>
<role-name>manager</role-name>
</auth-constraint>
</security-constraint>
<security-role>
<role-name>manager</role-name>
</security-role>
<login-config>
<auth-method>BASIC</auth-method>
</login-config>
</web-app>
Then I place a JkMount in my Apache httpd configuration (+ minimal
worker.properties):
JkMount /demo/* ajp13_worker
Testing this with curl works like expected:
root@1ae8973f1b6b:~# curl -I -v localhost/demo/
* Trying 127.0.0.1:80...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 80 (#0) > HEAD /demo/ HTTP/1.1 > Host: localhost > User-Agent: curl/7.68.0 > Accept: */* >
* Mark bundle as not supporting multiuse < HTTP/1.1 401 401
HTTP/1.1 401 401
< Date: Sat, 12 Feb 2022 12:57:33 GMT
Date: Sat, 12 Feb 2022 12:57:33 GMT
< Server: Apache/2.4.41 (Ubuntu)
Server: Apache/2.4.41 (Ubuntu)
< Cache-Control: private
Cache-Control: private
< WWW-Authenticate: Basic realm="Authentication required"
WWW-Authenticate: Basic realm="Authentication required"
< Content-Language: en
Content-Language: en
< Content-Type: text/html;charset=utf-8
Content-Type: text/html;charset=utf-8
<
* Connection #0 to host localhost left intact
But our default setup always includes custom error pages:
Alias /error/ "/usr/share/apache2/error/"
ErrorDocument 401 /error/HTTP_UNAUTHORIZED.html.var
If both of those lines are added this results in a response body for the HEAD request.
root@1ae8973f1b6b:~# curl -I -v localhost/demo/
* Trying 127.0.0.1:80...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 80 (#0) > HEAD /demo/ HTTP/1.1 > Host: localhost > User-Agent: curl/7.68.0 > Accept: */* >
* Mark bundle as not supporting multiuse < HTTP/1.1 401 401
HTTP/1.1 401 401
< Date: Sat, 12 Feb 2022 12:56:27 GMT
Date: Sat, 12 Feb 2022 12:56:27 GMT
< Server: Apache/2.4.41 (Ubuntu)
Server: Apache/2.4.41 (Ubuntu)
< Cache-Control: private
Cache-Control: private
< WWW-Authenticate: Basic realm="Authentication required"
WWW-Authenticate: Basic realm="Authentication required"
< Content-Language: en
Content-Language: en
< Content-Type: text/html;charset=utf-8
Content-Type: text/html;charset=utf-8
<
* Excess found: excess = 589 url = /demo/ (zero-length body)
* Connection #0 to host localhost left intact
Checking with tcpdump on port 8009 we see the expected response without a body from the Tomcat AJP connector. The tcpdump von port 80 reveals httpd is adding the configured ErrorDocument as response body.
If we comment out either the Alias or ErrorDocument directive the response is correct again.
Doing similar tests with CGI/PHP applications always show the correct response without a response body. This only affects requests which use mod_jk.
So far I could reproduce this on SLES12 SP5 and SLES15 SP3 running Apache httpd 2.4.51 and a self compile mod_jk 1.2.46 (same with the included mod_jk 1.2.43) at work. At home the same happens on a stock openSUSE Leap 15.3 with Apache httpd 2.4.51 and mod_jk 1.2.43 as well as on Ubuntu 20.04 with httpd 2.4.41 and mod_jk 1.2.46.
I didn't try to compile the latest mod_jk version yet because I didn't spot a relevant point in the changelog.
Can anyone confirm this behaviour or point me to a configuration directive i missed?
Thank you,
Stefan Mayr
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: mod_jk interference with ErrorDocument/Alias on HEAD request
Posted by Stefan Mayr <st...@mayr-stefan.de>.
Hi,
i created a bugzilla entry
(https://bz.apache.org/bugzilla/show_bug.cgi?id=65901) and included a
first attempt to patch this issue.
Am 12.02.2022 um 14:24 schrieb Stefan Mayr:
> Hello Tomcat users,
>
> this week we were debugging a strange connection issue which I tracked
> down to an interference between Apache httpd and mod_jk.
>
> For the full picture, the infrastructure setup contains
>
> 1. a Loadbalancer providing HTTPS, HTTP/1.1 and HTTP/2 for Clients.
> 2. an Apache httpd 2.4 webserver (HTTP only) with mod_jk
> 3. a Tomcat mit AJP-Connector
>
> We have an application doing many different HEAD requests against an
> application running in the Tomcat server. The requests contain an
> Authorization header for Basic authentication. Expected response is a
> HTTP 200 OK or HTTP 401 if this particular user is not allowed to access
> that ressource. Because this is a HEAD request there must not be a
> response body according to RFC 2616.
>
> If there is a response body in the response to the HEAD request our
> loadbalancer does strange things: aborts the connection if the clients
> uses HTTP/2, SSL errors if using HTTP/1.1. But this is an issue on its
> own which we might have to solve with the vendor.
>
> Now comes the point where I need your help. We have a httpd
> configuration with mod_jk which generates these invalid response bodies
> on HEAD requests. I have a gut feeling this could be a bug with mod_jk.
>
> For demonstration purpose i created a minimal demo app which only is a
> WEB-INF/web.xml
> <?xml version="1.0" encoding="UTF-8"?>
> <web-app xmlns="https://jakarta.ee/xml/ns/jakartaee"
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> xsi:schemaLocation="https://jakarta.ee/xml/ns/jakartaee
> https://jakarta.ee/xml/ns/jakartaee/web-app_5_0.xsd"
> version="5.0">
> <security-constraint>
> <web-resource-collection>
> <web-resource-name>Login</web-resource-name>
> <url-pattern>/*</url-pattern>
> </web-resource-collection>
> <auth-constraint>
> <role-name>manager</role-name>
> </auth-constraint>
> </security-constraint>
> <security-role>
> <role-name>manager</role-name>
> </security-role>
> <login-config>
> <auth-method>BASIC</auth-method>
> </login-config>
> </web-app>
>
> Then I place a JkMount in my Apache httpd configuration (+ minimal
> worker.properties):
>
> JkMount /demo/* ajp13_worker
>
> Testing this with curl works like expected:
>
> root@1ae8973f1b6b:~# curl -I -v localhost/demo/
> * Trying 127.0.0.1:80...
> * TCP_NODELAY set
> * Connected to localhost (127.0.0.1) port 80 (#0)
> > HEAD /demo/ HTTP/1.1
> > Host: localhost
> > User-Agent: curl/7.68.0
> > Accept: */*
> >
> * Mark bundle as not supporting multiuse
> < HTTP/1.1 401 401
> HTTP/1.1 401 401
> < Date: Sat, 12 Feb 2022 12:57:33 GMT
> Date: Sat, 12 Feb 2022 12:57:33 GMT
> < Server: Apache/2.4.41 (Ubuntu)
> Server: Apache/2.4.41 (Ubuntu)
> < Cache-Control: private
> Cache-Control: private
> < WWW-Authenticate: Basic realm="Authentication required"
> WWW-Authenticate: Basic realm="Authentication required"
> < Content-Language: en
> Content-Language: en
> < Content-Type: text/html;charset=utf-8
> Content-Type: text/html;charset=utf-8
>
> <
> * Connection #0 to host localhost left intact
>
> But our default setup always includes custom error pages:
>
> Alias /error/ "/usr/share/apache2/error/"
> ErrorDocument 401 /error/HTTP_UNAUTHORIZED.html.var
>
> If both of those lines are added this results in a response body for the
> HEAD request.
>
> root@1ae8973f1b6b:~# curl -I -v localhost/demo/
> * Trying 127.0.0.1:80...
> * TCP_NODELAY set
> * Connected to localhost (127.0.0.1) port 80 (#0)
> > HEAD /demo/ HTTP/1.1
> > Host: localhost
> > User-Agent: curl/7.68.0
> > Accept: */*
> >
> * Mark bundle as not supporting multiuse
> < HTTP/1.1 401 401
> HTTP/1.1 401 401
> < Date: Sat, 12 Feb 2022 12:56:27 GMT
> Date: Sat, 12 Feb 2022 12:56:27 GMT
> < Server: Apache/2.4.41 (Ubuntu)
> Server: Apache/2.4.41 (Ubuntu)
> < Cache-Control: private
> Cache-Control: private
> < WWW-Authenticate: Basic realm="Authentication required"
> WWW-Authenticate: Basic realm="Authentication required"
> < Content-Language: en
> Content-Language: en
> < Content-Type: text/html;charset=utf-8
> Content-Type: text/html;charset=utf-8
>
> <
> * Excess found: excess = 589 url = /demo/ (zero-length body)
> * Connection #0 to host localhost left intact
>
> Checking with tcpdump on port 8009 we see the expected response without
> a body from the Tomcat AJP connector. The tcpdump von port 80 reveals
> httpd is adding the configured ErrorDocument as response body.
>
> If we comment out either the Alias or ErrorDocument directive the
> response is correct again.
>
> Doing similar tests with CGI/PHP applications always show the correct
> response without a response body. This only affects requests which use
> mod_jk.
>
> So far I could reproduce this on SLES12 SP5 and SLES15 SP3 running
> Apache httpd 2.4.51 and a self compile mod_jk 1.2.46 (same with the
> included mod_jk 1.2.43) at work. At home the same happens on a stock
> openSUSE Leap 15.3 with Apache httpd 2.4.51 and mod_jk 1.2.43 as well as
> on Ubuntu 20.04 with httpd 2.4.41 and mod_jk 1.2.46.
> I didn't try to compile the latest mod_jk version yet because I didn't
> spot a relevant point in the changelog.
>
> Can anyone confirm this behaviour or point me to a configuration
> directive i missed?
>
> Thank you,
>
>
> Stefan Mayr
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
Regards,
Stefan Mayr
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
AW: mod_jk interference with ErrorDocument/Alias on HEAD request
Posted by "Thomas Hoffmann (Speed4Trade GmbH)" <Th...@speed4trade.com.INVALID>.
Hello Stefan,
Now I got it. Thanks for the clarification :)
-----Ursprüngliche Nachricht-----
Von: Stefan Mayr <st...@mayr-stefan.de>
Gesendet: Dienstag, 15. Februar 2022 14:26
An: users@tomcat.apache.org
Betreff: Re: mod_jk interference with ErrorDocument/Alias on HEAD request
Hello Thomas,
Am 15.02.2022 um 11:38 schrieb Thomas Hoffmann (Speed4Trade GmbH):
> Hello Stefan,
>
> by spec / RFC, a HEAD request is not allowed to return any body.
>
> Greetings,
> Thomas
This is true and that is why i'm writing to this list. In the described case mod_jk returns a response body although it should not (at least i think mod_jk is somehow responsible for that)
> -----Ursprüngliche Nachricht-----
> Von: Stefan Mayr <st...@mayr-stefan.de>
> Gesendet: Montag, 14. Februar 2022 23:07
> An: users@tomcat.apache.org
> Betreff: Re: mod_jk interference with ErrorDocument/Alias on HEAD
> request
>
> Hello again,
>
> a self-compiled mod_jk 1.2.48 shows the same issue.
>
> Am 13.02.2022 um 18:37 schrieb Stefan Mayr:
>> Hi,
>>
>> looking at the source code
>> https://github.com/apache/tomcat-connectors/blob/main/native/apache-2.
>> 0/mod_jk.c#L2954#L2973
>> I did some more testing:
>>
>> Variant 1: JkMount /demo/* ajp13_worker;use_server_errors=401
>> Variant 2: JkMount /demo/* ajp13_worker
>>
>> ignoring what variant 2 changes for regular request: reading the
>> source comment my understanding is, that for both variants a HEAD
>> request (by definition must have an empty response body) should let
>> Apache httpd handle the error code.
>>
>> But the return code for jk_handler looks different:
>>
>> Variant 1: s.http_response_status
>> Variant 2: r->status
>
> Although this looks different on the first glance it seems to be the same.
>
>> The response only seems correct for variant 1 - which is configured
>> to let Apache httpd handle all responses for status codes >= 401. For
>> variant 2 mod_jk seems to handle the response itself - contrary to
>> what the comment explains.
>
> This leads to the next assumption, that whenever there is a special handling for use_server_errors there should be something similar for the case with an empty/non-existing response body.
>
> There is
> https://github.com/apache/tomcat-connectors/blob/main/native/common/jk
> _ajp_common.c#L1991#L1993 with no corresponding (pseudo) code like
>
> if (!r->something_like_bodyct && r->http_response_status >= JK_HTTP_BAD_REQUEST){
> r->response_blocked = JK_TRUE;
> }
>
> Adding code like this (sorry, i could not find out how to determine if there is a response body) fixes the issue with the wrong response body for a HEAD request. But we miss the WWW-Authenticate header now.
>
> Digging further we find
> https://github.com/apache/tomcat-connectors/blob/main/native/apache-2.
> 0/mod_jk.c#L331#L353 which has a special treatment for 401 HTTP
> Unauthorized. But again, only for use_server_errors.
>
> This should be fixable by extending the condition like this
>
> if ((s->extension.use_server_error_pages &&
> status >= s->extension.use_server_error_pages) ||
> (!r->sent_bodyct && r->status >= HTTP_BAD_REQUEST)) {
>
>
> But the WWW-Authenticate header is still missing. So i'm wrong, again.
> Although it feels like i'm close.
>
>> Am 12.02.2022 um 14:24 schrieb Stefan Mayr:
>>> Hello Tomcat users,
>>>
>>> this week we were debugging a strange connection issue which I
>>> tracked down to an interference between Apache httpd and mod_jk.
>>>
>>> For the full picture, the infrastructure setup contains
>>>
>>> 1. a Loadbalancer providing HTTPS, HTTP/1.1 and HTTP/2 for Clients.
>>> 2. an Apache httpd 2.4 webserver (HTTP only) with mod_jk 3. a Tomcat
>>> mit AJP-Connector
>>>
>>> We have an application doing many different HEAD requests against an
>>> application running in the Tomcat server. The requests contain an
>>> Authorization header for Basic authentication. Expected response is
>>> a HTTP 200 OK or HTTP 401 if this particular user is not allowed to
>>> access that ressource. Because this is a HEAD request there must not
>>> be a response body according to RFC 2616.
>>>
>>> If there is a response body in the response to the HEAD request our
>>> loadbalancer does strange things: aborts the connection if the
>>> clients uses HTTP/2, SSL errors if using HTTP/1.1. But this is an
>>> issue on its own which we might have to solve with the vendor.
>>>
>>> Now comes the point where I need your help. We have a httpd
>>> configuration with mod_jk which generates these invalid response
>>> bodies on HEAD requests. I have a gut feeling this could be a bug
>>> with mod_jk.
>>>
>>> For demonstration purpose i created a minimal demo app which only is
>>> a WEB-INF/web.xml <?xml version="1.0" encoding="UTF-8"?> <web-app
>>> xmlns="https://jakarta.ee/xml/ns/jakartaee"
>>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>>> xsi:schemaLocation="https://jakarta.ee/xml/ns/jakartaee
>>>
>>> https://jakarta.ee/xml/ns/jakartaee/web-app_5_0.xsd"
>>> version="5.0">
>>> <security-constraint>
>>> <web-resource-collection>
>>>
>>> <web-resource-name>Login</web-resource-name>
>>> <url-pattern>/*</url-pattern>
>>> </web-resource-collection>
>>> <auth-constraint>
>>> <role-name>manager</role-name>
>>> </auth-constraint>
>>> </security-constraint>
>>> <security-role>
>>> <role-name>manager</role-name>
>>> </security-role>
>>> <login-config>
>>> <auth-method>BASIC</auth-method>
>>> </login-config>
>>> </web-app>
>>>
>>> Then I place a JkMount in my Apache httpd configuration (+ minimal
>>> worker.properties):
>>>
>>> JkMount /demo/* ajp13_worker
>>>
>>> Testing this with curl works like expected:
>>>
>>> root@1ae8973f1b6b:~# curl -I -v localhost/demo/
>>> * Trying 127.0.0.1:80...
>>> * TCP_NODELAY set
>>> * Connected to localhost (127.0.0.1) port 80 (#0)
>>> > HEAD /demo/ HTTP/1.1
>>> > Host: localhost
>>> > User-Agent: curl/7.68.0
>>> > Accept: */*
>>> >
>>> * Mark bundle as not supporting multiuse < HTTP/1.1 401 401
>>> HTTP/1.1 401 401
>>> < Date: Sat, 12 Feb 2022 12:57:33 GMT
>>> Date: Sat, 12 Feb 2022 12:57:33 GMT
>>> < Server: Apache/2.4.41 (Ubuntu)
>>> Server: Apache/2.4.41 (Ubuntu)
>>> < Cache-Control: private
>>> Cache-Control: private
>>> < WWW-Authenticate: Basic realm="Authentication required"
>>> WWW-Authenticate: Basic realm="Authentication required"
>>> < Content-Language: en
>>> Content-Language: en
>>> < Content-Type: text/html;charset=utf-8
>>> Content-Type: text/html;charset=utf-8
>>>
>>> <
>>> * Connection #0 to host localhost left intact
>>>
>>> But our default setup always includes custom error pages:
>>>
>>> Alias /error/ "/usr/share/apache2/error/"
>>> ErrorDocument 401 /error/HTTP_UNAUTHORIZED.html.var
>>>
>>> If both of those lines are added this results in a response body for
>>> the HEAD request.
>>>
>>> root@1ae8973f1b6b:~# curl -I -v localhost/demo/
>>> * Trying 127.0.0.1:80...
>>> * TCP_NODELAY set
>>> * Connected to localhost (127.0.0.1) port 80 (#0)
>>> > HEAD /demo/ HTTP/1.1
>>> > Host: localhost
>>> > User-Agent: curl/7.68.0
>>> > Accept: */*
>>> >
>>> * Mark bundle as not supporting multiuse < HTTP/1.1 401 401
>>> HTTP/1.1 401 401
>>> < Date: Sat, 12 Feb 2022 12:56:27 GMT
>>> Date: Sat, 12 Feb 2022 12:56:27 GMT
>>> < Server: Apache/2.4.41 (Ubuntu)
>>> Server: Apache/2.4.41 (Ubuntu)
>>> < Cache-Control: private
>>> Cache-Control: private
>>> < WWW-Authenticate: Basic realm="Authentication required"
>>> WWW-Authenticate: Basic realm="Authentication required"
>>> < Content-Language: en
>>> Content-Language: en
>>> < Content-Type: text/html;charset=utf-8
>>> Content-Type: text/html;charset=utf-8
>>>
>>> <
>>> * Excess found: excess = 589 url = /demo/ (zero-length body)
>>> * Connection #0 to host localhost left intact
>>>
>>> Checking with tcpdump on port 8009 we see the expected response
>>> without a body from the Tomcat AJP connector. The tcpdump von port
>>> 80 reveals httpd is adding the configured ErrorDocument as response body.
>>>
>>> If we comment out either the Alias or ErrorDocument directive the
>>> response is correct again.
>>>
>>> Doing similar tests with CGI/PHP applications always show the
>>> correct response without a response body. This only affects requests
>>> which use mod_jk.
>>>
>>> So far I could reproduce this on SLES12 SP5 and SLES15 SP3 running
>>> Apache httpd 2.4.51 and a self compile mod_jk 1.2.46 (same with the
>>> included mod_jk 1.2.43) at work. At home the same happens on a stock
>>> openSUSE Leap 15.3 with Apache httpd 2.4.51 and mod_jk 1.2.43 as
>>> well as on Ubuntu 20.04 with httpd 2.4.41 and mod_jk 1.2.46.
>>> I didn't try to compile the latest mod_jk version yet because I
>>> didn't spot a relevant point in the changelog.
>>>
>>> Can anyone confirm this behaviour or point me to a configuration
>>>directive i missed?
>>>--
Regards,
Stefan Mayr
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
AW: mod_jk interference with ErrorDocument/Alias on HEAD request
Posted by "Thomas Hoffmann (Speed4Trade GmbH)" <Th...@speed4trade.com.INVALID>.
Hello Stefan,
the debug output of mod_jk shows at least which route the request is going:
[info] jk_handler::mod_jk.c (2968): No body with status=401 for worker=ajp13_worker
So it looks like that the code https://github.com/apache/tomcat-connectors/blob/main/native/apache-2.0/mod_jk.c#L2954#L2973
Returns 401 to the caller.
Apache sets the flag header_only when receiving a HEAD-Request. This flag can also be seen in the mod_jk sources.
The "Excess" error is produces by curl: https://fossies.org/linux/curl/lib/transfer.c
Dunno if this info helps much.
Greetings, Thomas
-----Ursprüngliche Nachricht-----
Von: Stefan Mayr <st...@mayr-stefan.de>
Gesendet: Dienstag, 15. Februar 2022 14:26
An: users@tomcat.apache.org
Betreff: Re: mod_jk interference with ErrorDocument/Alias on HEAD request
Hello Thomas,
Am 15.02.2022 um 11:38 schrieb Thomas Hoffmann (Speed4Trade GmbH):
> Hello Stefan,
>
> by spec / RFC, a HEAD request is not allowed to return any body.
>
> Greetings,
> Thomas
This is true and that is why i'm writing to this list. In the described case mod_jk returns a response body although it should not (at least i think mod_jk is somehow responsible for that)
> -----Ursprüngliche Nachricht-----
> Von: Stefan Mayr <st...@mayr-stefan.de>
> Gesendet: Montag, 14. Februar 2022 23:07
> An: users@tomcat.apache.org
> Betreff: Re: mod_jk interference with ErrorDocument/Alias on HEAD
> request
>
> Hello again,
>
> a self-compiled mod_jk 1.2.48 shows the same issue.
>
> Am 13.02.2022 um 18:37 schrieb Stefan Mayr:
>> Hi,
>>
>> looking at the source code
>> https://github.com/apache/tomcat-connectors/blob/main/native/apache-2.
>> 0/mod_jk.c#L2954#L2973
>> I did some more testing:
>>
>> Variant 1: JkMount /demo/* ajp13_worker;use_server_errors=401
>> Variant 2: JkMount /demo/* ajp13_worker
>>
>> ignoring what variant 2 changes for regular request: reading the
>> source comment my understanding is, that for both variants a HEAD
>> request (by definition must have an empty response body) should let
>> Apache httpd handle the error code.
>>
>> But the return code for jk_handler looks different:
>>
>> Variant 1: s.http_response_status
>> Variant 2: r->status
>
> Although this looks different on the first glance it seems to be the same.
>
>> The response only seems correct for variant 1 - which is configured
>> to let Apache httpd handle all responses for status codes >= 401. For
>> variant 2 mod_jk seems to handle the response itself - contrary to
>> what the comment explains.
>
> This leads to the next assumption, that whenever there is a special handling for use_server_errors there should be something similar for the case with an empty/non-existing response body.
>
> There is
> https://github.com/apache/tomcat-connectors/blob/main/native/common/jk
> _ajp_common.c#L1991#L1993 with no corresponding (pseudo) code like
>
> if (!r->something_like_bodyct && r->http_response_status >= JK_HTTP_BAD_REQUEST){
> r->response_blocked = JK_TRUE;
> }
>
> Adding code like this (sorry, i could not find out how to determine if there is a response body) fixes the issue with the wrong response body for a HEAD request. But we miss the WWW-Authenticate header now.
>
> Digging further we find
> https://github.com/apache/tomcat-connectors/blob/main/native/apache-2.
> 0/mod_jk.c#L331#L353 which has a special treatment for 401 HTTP
> Unauthorized. But again, only for use_server_errors.
>
> This should be fixable by extending the condition like this
>
> if ((s->extension.use_server_error_pages &&
> status >= s->extension.use_server_error_pages) ||
> (!r->sent_bodyct && r->status >= HTTP_BAD_REQUEST)) {
>
>
> But the WWW-Authenticate header is still missing. So i'm wrong, again.
> Although it feels like i'm close.
>
>> Am 12.02.2022 um 14:24 schrieb Stefan Mayr:
>>> Hello Tomcat users,
>>>
>>> this week we were debugging a strange connection issue which I
>>> tracked down to an interference between Apache httpd and mod_jk.
>>>
>>> For the full picture, the infrastructure setup contains
>>>
>>> 1. a Loadbalancer providing HTTPS, HTTP/1.1 and HTTP/2 for Clients.
>>> 2. an Apache httpd 2.4 webserver (HTTP only) with mod_jk 3. a Tomcat
>>> mit AJP-Connector
>>>
>>> We have an application doing many different HEAD requests against an
>>> application running in the Tomcat server. The requests contain an
>>> Authorization header for Basic authentication. Expected response is
>>> a HTTP 200 OK or HTTP 401 if this particular user is not allowed to
>>> access that ressource. Because this is a HEAD request there must not
>>> be a response body according to RFC 2616.
>>>
>>> If there is a response body in the response to the HEAD request our
>>> loadbalancer does strange things: aborts the connection if the
>>> clients uses HTTP/2, SSL errors if using HTTP/1.1. But this is an
>>> issue on its own which we might have to solve with the vendor.
>>>
>>> Now comes the point where I need your help. We have a httpd
>>> configuration with mod_jk which generates these invalid response
>>> bodies on HEAD requests. I have a gut feeling this could be a bug
>>> with mod_jk.
>>>
>>> For demonstration purpose i created a minimal demo app which only is
>>> a WEB-INF/web.xml <?xml version="1.0" encoding="UTF-8"?> <web-app
>>> xmlns="https://jakarta.ee/xml/ns/jakartaee"
>>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>>> xsi:schemaLocation="https://jakarta.ee/xml/ns/jakartaee
>>>
>>> https://jakarta.ee/xml/ns/jakartaee/web-app_5_0.xsd"
>>> version="5.0">
>>> <security-constraint>
>>> <web-resource-collection>
>>>
>>> <web-resource-name>Login</web-resource-name>
>>> <url-pattern>/*</url-pattern>
>>> </web-resource-collection>
>>> <auth-constraint>
>>> <role-name>manager</role-name>
>>> </auth-constraint>
>>> </security-constraint>
>>> <security-role>
>>> <role-name>manager</role-name>
>>> </security-role>
>>> <login-config>
>>> <auth-method>BASIC</auth-method>
>>> </login-config>
>>> </web-app>
>>>
>>> Then I place a JkMount in my Apache httpd configuration (+ minimal
>>> worker.properties):
>>>
>>> JkMount /demo/* ajp13_worker
>>>
>>> Testing this with curl works like expected:
>>>
>>> root@1ae8973f1b6b:~# curl -I -v localhost/demo/
>>> * Trying 127.0.0.1:80...
>>> * TCP_NODELAY set
>>> * Connected to localhost (127.0.0.1) port 80 (#0)
>>> > HEAD /demo/ HTTP/1.1
>>> > Host: localhost
>>> > User-Agent: curl/7.68.0
>>> > Accept: */*
>>> >
>>> * Mark bundle as not supporting multiuse < HTTP/1.1 401 401
>>> HTTP/1.1 401 401
>>> < Date: Sat, 12 Feb 2022 12:57:33 GMT
>>> Date: Sat, 12 Feb 2022 12:57:33 GMT
>>> < Server: Apache/2.4.41 (Ubuntu)
>>> Server: Apache/2.4.41 (Ubuntu)
>>> < Cache-Control: private
>>> Cache-Control: private
>>> < WWW-Authenticate: Basic realm="Authentication required"
>>> WWW-Authenticate: Basic realm="Authentication required"
>>> < Content-Language: en
>>> Content-Language: en
>>> < Content-Type: text/html;charset=utf-8
>>> Content-Type: text/html;charset=utf-8
>>>
>>> <
>>> * Connection #0 to host localhost left intact
>>>
>>> But our default setup always includes custom error pages:
>>>
>>> Alias /error/ "/usr/share/apache2/error/"
>>> ErrorDocument 401 /error/HTTP_UNAUTHORIZED.html.var
>>>
>>> If both of those lines are added this results in a response body for
>>> the HEAD request.
>>>
>>> root@1ae8973f1b6b:~# curl -I -v localhost/demo/
>>> * Trying 127.0.0.1:80...
>>> * TCP_NODELAY set
>>> * Connected to localhost (127.0.0.1) port 80 (#0)
>>> > HEAD /demo/ HTTP/1.1
>>> > Host: localhost
>>> > User-Agent: curl/7.68.0
>>> > Accept: */*
>>> >
>>> * Mark bundle as not supporting multiuse < HTTP/1.1 401 401
>>> HTTP/1.1 401 401
>>> < Date: Sat, 12 Feb 2022 12:56:27 GMT
>>> Date: Sat, 12 Feb 2022 12:56:27 GMT
>>> < Server: Apache/2.4.41 (Ubuntu)
>>> Server: Apache/2.4.41 (Ubuntu)
>>> < Cache-Control: private
>>> Cache-Control: private
>>> < WWW-Authenticate: Basic realm="Authentication required"
>>> WWW-Authenticate: Basic realm="Authentication required"
>>> < Content-Language: en
>>> Content-Language: en
>>> < Content-Type: text/html;charset=utf-8
>>> Content-Type: text/html;charset=utf-8
>>>
>>> <
>>> * Excess found: excess = 589 url = /demo/ (zero-length body)
>>> * Connection #0 to host localhost left intact
>>>
>>> Checking with tcpdump on port 8009 we see the expected response
>>> without a body from the Tomcat AJP connector. The tcpdump von port
>>> 80 reveals httpd is adding the configured ErrorDocument as response body.
>>>
>>> If we comment out either the Alias or ErrorDocument directive the
>>> response is correct again.
>>>
>>> Doing similar tests with CGI/PHP applications always show the
>>> correct response without a response body. This only affects requests
>>> which use mod_jk.
>>>
>>> So far I could reproduce this on SLES12 SP5 and SLES15 SP3 running
>>> Apache httpd 2.4.51 and a self compile mod_jk 1.2.46 (same with the
>>> included mod_jk 1.2.43) at work. At home the same happens on a stock
>>> openSUSE Leap 15.3 with Apache httpd 2.4.51 and mod_jk 1.2.43 as
>>> well as on Ubuntu 20.04 with httpd 2.4.41 and mod_jk 1.2.46.
>>> I didn't try to compile the latest mod_jk version yet because I
>>> didn't spot a relevant point in the changelog.
>>>
>>> Can anyone confirm this behaviour or point me to a configuration
>>>directive i missed?
>>>--
Regards,
Stefan Mayr
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: mod_jk interference with ErrorDocument/Alias on HEAD request
Posted by Stefan Mayr <st...@mayr-stefan.de>.
Hello Thomas,
Am 15.02.2022 um 11:38 schrieb Thomas Hoffmann (Speed4Trade GmbH):
> Hello Stefan,
>
> by spec / RFC, a HEAD request is not allowed to return any body.
>
> Greetings,
> Thomas
This is true and that is why i'm writing to this list. In the described
case mod_jk returns a response body although it should not (at least i
think mod_jk is somehow responsible for that)
> -----Ursprüngliche Nachricht-----
> Von: Stefan Mayr <st...@mayr-stefan.de>
> Gesendet: Montag, 14. Februar 2022 23:07
> An: users@tomcat.apache.org
> Betreff: Re: mod_jk interference with ErrorDocument/Alias on HEAD request
>
> Hello again,
>
> a self-compiled mod_jk 1.2.48 shows the same issue.
>
> Am 13.02.2022 um 18:37 schrieb Stefan Mayr:
>> Hi,
>>
>> looking at the source code
>> https://github.com/apache/tomcat-connectors/blob/main/native/apache-2.
>> 0/mod_jk.c#L2954#L2973
>> I did some more testing:
>>
>> Variant 1: JkMount /demo/* ajp13_worker;use_server_errors=401
>> Variant 2: JkMount /demo/* ajp13_worker
>>
>> ignoring what variant 2 changes for regular request: reading the
>> source comment my understanding is, that for both variants a HEAD
>> request (by definition must have an empty response body) should let
>> Apache httpd handle the error code.
>>
>> But the return code for jk_handler looks different:
>>
>> Variant 1: s.http_response_status
>> Variant 2: r->status
>
> Although this looks different on the first glance it seems to be the same.
>
>> The response only seems correct for variant 1 - which is configured to
>> let Apache httpd handle all responses for status codes >= 401. For
>> variant 2 mod_jk seems to handle the response itself - contrary to
>> what the comment explains.
>
> This leads to the next assumption, that whenever there is a special handling for use_server_errors there should be something similar for the case with an empty/non-existing response body.
>
> There is
> https://github.com/apache/tomcat-connectors/blob/main/native/common/jk_ajp_common.c#L1991#L1993
> with no corresponding (pseudo) code like
>
> if (!r->something_like_bodyct && r->http_response_status >= JK_HTTP_BAD_REQUEST){
> r->response_blocked = JK_TRUE;
> }
>
> Adding code like this (sorry, i could not find out how to determine if there is a response body) fixes the issue with the wrong response body for a HEAD request. But we miss the WWW-Authenticate header now.
>
> Digging further we find
> https://github.com/apache/tomcat-connectors/blob/main/native/apache-2.0/mod_jk.c#L331#L353
> which has a special treatment for 401 HTTP Unauthorized. But again, only for use_server_errors.
>
> This should be fixable by extending the condition like this
>
> if ((s->extension.use_server_error_pages &&
> status >= s->extension.use_server_error_pages) ||
> (!r->sent_bodyct && r->status >= HTTP_BAD_REQUEST)) {
>
>
> But the WWW-Authenticate header is still missing. So i'm wrong, again.
> Although it feels like i'm close.
>
>> Am 12.02.2022 um 14:24 schrieb Stefan Mayr:
>>> Hello Tomcat users,
>>>
>>> this week we were debugging a strange connection issue which I
>>> tracked down to an interference between Apache httpd and mod_jk.
>>>
>>> For the full picture, the infrastructure setup contains
>>>
>>> 1. a Loadbalancer providing HTTPS, HTTP/1.1 and HTTP/2 for Clients.
>>> 2. an Apache httpd 2.4 webserver (HTTP only) with mod_jk 3. a Tomcat
>>> mit AJP-Connector
>>>
>>> We have an application doing many different HEAD requests against an
>>> application running in the Tomcat server. The requests contain an
>>> Authorization header for Basic authentication. Expected response is a
>>> HTTP 200 OK or HTTP 401 if this particular user is not allowed to
>>> access that ressource. Because this is a HEAD request there must not
>>> be a response body according to RFC 2616.
>>>
>>> If there is a response body in the response to the HEAD request our
>>> loadbalancer does strange things: aborts the connection if the
>>> clients uses HTTP/2, SSL errors if using HTTP/1.1. But this is an
>>> issue on its own which we might have to solve with the vendor.
>>>
>>> Now comes the point where I need your help. We have a httpd
>>> configuration with mod_jk which generates these invalid response
>>> bodies on HEAD requests. I have a gut feeling this could be a bug
>>> with mod_jk.
>>>
>>> For demonstration purpose i created a minimal demo app which only is
>>> a WEB-INF/web.xml <?xml version="1.0" encoding="UTF-8"?> <web-app
>>> xmlns="https://jakarta.ee/xml/ns/jakartaee"
>>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>>> xsi:schemaLocation="https://jakarta.ee/xml/ns/jakartaee
>>>
>>> https://jakarta.ee/xml/ns/jakartaee/web-app_5_0.xsd"
>>> version="5.0">
>>> <security-constraint>
>>> <web-resource-collection>
>>> <web-resource-name>Login</web-resource-name>
>>> <url-pattern>/*</url-pattern>
>>> </web-resource-collection>
>>> <auth-constraint>
>>> <role-name>manager</role-name>
>>> </auth-constraint>
>>> </security-constraint>
>>> <security-role>
>>> <role-name>manager</role-name>
>>> </security-role>
>>> <login-config>
>>> <auth-method>BASIC</auth-method>
>>> </login-config>
>>> </web-app>
>>>
>>> Then I place a JkMount in my Apache httpd configuration (+ minimal
>>> worker.properties):
>>>
>>> JkMount /demo/* ajp13_worker
>>>
>>> Testing this with curl works like expected:
>>>
>>> root@1ae8973f1b6b:~# curl -I -v localhost/demo/
>>> * Trying 127.0.0.1:80...
>>> * TCP_NODELAY set
>>> * Connected to localhost (127.0.0.1) port 80 (#0)
>>> > HEAD /demo/ HTTP/1.1
>>> > Host: localhost
>>> > User-Agent: curl/7.68.0
>>> > Accept: */*
>>> >
>>> * Mark bundle as not supporting multiuse < HTTP/1.1 401 401
>>> HTTP/1.1 401 401
>>> < Date: Sat, 12 Feb 2022 12:57:33 GMT
>>> Date: Sat, 12 Feb 2022 12:57:33 GMT
>>> < Server: Apache/2.4.41 (Ubuntu)
>>> Server: Apache/2.4.41 (Ubuntu)
>>> < Cache-Control: private
>>> Cache-Control: private
>>> < WWW-Authenticate: Basic realm="Authentication required"
>>> WWW-Authenticate: Basic realm="Authentication required"
>>> < Content-Language: en
>>> Content-Language: en
>>> < Content-Type: text/html;charset=utf-8
>>> Content-Type: text/html;charset=utf-8
>>>
>>> <
>>> * Connection #0 to host localhost left intact
>>>
>>> But our default setup always includes custom error pages:
>>>
>>> Alias /error/ "/usr/share/apache2/error/"
>>> ErrorDocument 401 /error/HTTP_UNAUTHORIZED.html.var
>>>
>>> If both of those lines are added this results in a response body for
>>> the HEAD request.
>>>
>>> root@1ae8973f1b6b:~# curl -I -v localhost/demo/
>>> * Trying 127.0.0.1:80...
>>> * TCP_NODELAY set
>>> * Connected to localhost (127.0.0.1) port 80 (#0)
>>> > HEAD /demo/ HTTP/1.1
>>> > Host: localhost
>>> > User-Agent: curl/7.68.0
>>> > Accept: */*
>>> >
>>> * Mark bundle as not supporting multiuse < HTTP/1.1 401 401
>>> HTTP/1.1 401 401
>>> < Date: Sat, 12 Feb 2022 12:56:27 GMT
>>> Date: Sat, 12 Feb 2022 12:56:27 GMT
>>> < Server: Apache/2.4.41 (Ubuntu)
>>> Server: Apache/2.4.41 (Ubuntu)
>>> < Cache-Control: private
>>> Cache-Control: private
>>> < WWW-Authenticate: Basic realm="Authentication required"
>>> WWW-Authenticate: Basic realm="Authentication required"
>>> < Content-Language: en
>>> Content-Language: en
>>> < Content-Type: text/html;charset=utf-8
>>> Content-Type: text/html;charset=utf-8
>>>
>>> <
>>> * Excess found: excess = 589 url = /demo/ (zero-length body)
>>> * Connection #0 to host localhost left intact
>>>
>>> Checking with tcpdump on port 8009 we see the expected response
>>> without a body from the Tomcat AJP connector. The tcpdump von port 80
>>> reveals httpd is adding the configured ErrorDocument as response body.
>>>
>>> If we comment out either the Alias or ErrorDocument directive the
>>> response is correct again.
>>>
>>> Doing similar tests with CGI/PHP applications always show the correct
>>> response without a response body. This only affects requests which
>>> use mod_jk.
>>>
>>> So far I could reproduce this on SLES12 SP5 and SLES15 SP3 running
>>> Apache httpd 2.4.51 and a self compile mod_jk 1.2.46 (same with the
>>> included mod_jk 1.2.43) at work. At home the same happens on a stock
>>> openSUSE Leap 15.3 with Apache httpd 2.4.51 and mod_jk 1.2.43 as well
>>> as on Ubuntu 20.04 with httpd 2.4.41 and mod_jk 1.2.46.
>>> I didn't try to compile the latest mod_jk version yet because I
>>> didn't spot a relevant point in the changelog.
>>>
>>> Can anyone confirm this behaviour or point me to a configuration
>>> directive i missed?
>>>--
Regards,
Stefan Mayr
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
AW: mod_jk interference with ErrorDocument/Alias on HEAD request
Posted by "Thomas Hoffmann (Speed4Trade GmbH)" <Th...@speed4trade.com.INVALID>.
Hello Stefan,
by spec / RFC, a HEAD request is not allowed to return any body.
Greetings,
Thomas
-----Ursprüngliche Nachricht-----
Von: Stefan Mayr <st...@mayr-stefan.de>
Gesendet: Montag, 14. Februar 2022 23:07
An: users@tomcat.apache.org
Betreff: Re: mod_jk interference with ErrorDocument/Alias on HEAD request
Hello again,
a self-compiled mod_jk 1.2.48 shows the same issue.
Am 13.02.2022 um 18:37 schrieb Stefan Mayr:
> Hi,
>
> looking at the source code
> https://github.com/apache/tomcat-connectors/blob/main/native/apache-2.
> 0/mod_jk.c#L2954#L2973
> I did some more testing:
>
> Variant 1: JkMount /demo/* ajp13_worker;use_server_errors=401
> Variant 2: JkMount /demo/* ajp13_worker
>
> ignoring what variant 2 changes for regular request: reading the
> source comment my understanding is, that for both variants a HEAD
> request (by definition must have an empty response body) should let
> Apache httpd handle the error code.
>
> But the return code for jk_handler looks different:
>
> Variant 1: s.http_response_status
> Variant 2: r->status
Although this looks different on the first glance it seems to be the same.
> The response only seems correct for variant 1 - which is configured to
> let Apache httpd handle all responses for status codes >= 401. For
> variant 2 mod_jk seems to handle the response itself - contrary to
> what the comment explains.
This leads to the next assumption, that whenever there is a special handling for use_server_errors there should be something similar for the case with an empty/non-existing response body.
There is
https://github.com/apache/tomcat-connectors/blob/main/native/common/jk_ajp_common.c#L1991#L1993
with no corresponding (pseudo) code like
if (!r->something_like_bodyct && r->http_response_status >= JK_HTTP_BAD_REQUEST){
r->response_blocked = JK_TRUE;
}
Adding code like this (sorry, i could not find out how to determine if there is a response body) fixes the issue with the wrong response body for a HEAD request. But we miss the WWW-Authenticate header now.
Digging further we find
https://github.com/apache/tomcat-connectors/blob/main/native/apache-2.0/mod_jk.c#L331#L353
which has a special treatment for 401 HTTP Unauthorized. But again, only for use_server_errors.
This should be fixable by extending the condition like this
if ((s->extension.use_server_error_pages &&
status >= s->extension.use_server_error_pages) ||
(!r->sent_bodyct && r->status >= HTTP_BAD_REQUEST)) {
But the WWW-Authenticate header is still missing. So i'm wrong, again.
Although it feels like i'm close.
> Am 12.02.2022 um 14:24 schrieb Stefan Mayr:
>> Hello Tomcat users,
>>
>> this week we were debugging a strange connection issue which I
>> tracked down to an interference between Apache httpd and mod_jk.
>>
>> For the full picture, the infrastructure setup contains
>>
>> 1. a Loadbalancer providing HTTPS, HTTP/1.1 and HTTP/2 for Clients.
>> 2. an Apache httpd 2.4 webserver (HTTP only) with mod_jk 3. a Tomcat
>> mit AJP-Connector
>>
>> We have an application doing many different HEAD requests against an
>> application running in the Tomcat server. The requests contain an
>> Authorization header for Basic authentication. Expected response is a
>> HTTP 200 OK or HTTP 401 if this particular user is not allowed to
>> access that ressource. Because this is a HEAD request there must not
>> be a response body according to RFC 2616.
>>
>> If there is a response body in the response to the HEAD request our
>> loadbalancer does strange things: aborts the connection if the
>> clients uses HTTP/2, SSL errors if using HTTP/1.1. But this is an
>> issue on its own which we might have to solve with the vendor.
>>
>> Now comes the point where I need your help. We have a httpd
>> configuration with mod_jk which generates these invalid response
>> bodies on HEAD requests. I have a gut feeling this could be a bug
>> with mod_jk.
>>
>> For demonstration purpose i created a minimal demo app which only is
>> a WEB-INF/web.xml <?xml version="1.0" encoding="UTF-8"?> <web-app
>> xmlns="https://jakarta.ee/xml/ns/jakartaee"
>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>> xsi:schemaLocation="https://jakarta.ee/xml/ns/jakartaee
>>
>> https://jakarta.ee/xml/ns/jakartaee/web-app_5_0.xsd"
>> version="5.0">
>> <security-constraint>
>> <web-resource-collection>
>> <web-resource-name>Login</web-resource-name>
>> <url-pattern>/*</url-pattern>
>> </web-resource-collection>
>> <auth-constraint>
>> <role-name>manager</role-name>
>> </auth-constraint>
>> </security-constraint>
>> <security-role>
>> <role-name>manager</role-name>
>> </security-role>
>> <login-config>
>> <auth-method>BASIC</auth-method>
>> </login-config>
>> </web-app>
>>
>> Then I place a JkMount in my Apache httpd configuration (+ minimal
>> worker.properties):
>>
>> JkMount /demo/* ajp13_worker
>>
>> Testing this with curl works like expected:
>>
>> root@1ae8973f1b6b:~# curl -I -v localhost/demo/
>> * Trying 127.0.0.1:80...
>> * TCP_NODELAY set
>> * Connected to localhost (127.0.0.1) port 80 (#0)
>> > HEAD /demo/ HTTP/1.1
>> > Host: localhost
>> > User-Agent: curl/7.68.0
>> > Accept: */*
>> >
>> * Mark bundle as not supporting multiuse < HTTP/1.1 401 401
>> HTTP/1.1 401 401
>> < Date: Sat, 12 Feb 2022 12:57:33 GMT
>> Date: Sat, 12 Feb 2022 12:57:33 GMT
>> < Server: Apache/2.4.41 (Ubuntu)
>> Server: Apache/2.4.41 (Ubuntu)
>> < Cache-Control: private
>> Cache-Control: private
>> < WWW-Authenticate: Basic realm="Authentication required"
>> WWW-Authenticate: Basic realm="Authentication required"
>> < Content-Language: en
>> Content-Language: en
>> < Content-Type: text/html;charset=utf-8
>> Content-Type: text/html;charset=utf-8
>>
>> <
>> * Connection #0 to host localhost left intact
>>
>> But our default setup always includes custom error pages:
>>
>> Alias /error/ "/usr/share/apache2/error/"
>> ErrorDocument 401 /error/HTTP_UNAUTHORIZED.html.var
>>
>> If both of those lines are added this results in a response body for
>> the HEAD request.
>>
>> root@1ae8973f1b6b:~# curl -I -v localhost/demo/
>> * Trying 127.0.0.1:80...
>> * TCP_NODELAY set
>> * Connected to localhost (127.0.0.1) port 80 (#0)
>> > HEAD /demo/ HTTP/1.1
>> > Host: localhost
>> > User-Agent: curl/7.68.0
>> > Accept: */*
>> >
>> * Mark bundle as not supporting multiuse < HTTP/1.1 401 401
>> HTTP/1.1 401 401
>> < Date: Sat, 12 Feb 2022 12:56:27 GMT
>> Date: Sat, 12 Feb 2022 12:56:27 GMT
>> < Server: Apache/2.4.41 (Ubuntu)
>> Server: Apache/2.4.41 (Ubuntu)
>> < Cache-Control: private
>> Cache-Control: private
>> < WWW-Authenticate: Basic realm="Authentication required"
>> WWW-Authenticate: Basic realm="Authentication required"
>> < Content-Language: en
>> Content-Language: en
>> < Content-Type: text/html;charset=utf-8
>> Content-Type: text/html;charset=utf-8
>>
>> <
>> * Excess found: excess = 589 url = /demo/ (zero-length body)
>> * Connection #0 to host localhost left intact
>>
>> Checking with tcpdump on port 8009 we see the expected response
>> without a body from the Tomcat AJP connector. The tcpdump von port 80
>> reveals httpd is adding the configured ErrorDocument as response body.
>>
>> If we comment out either the Alias or ErrorDocument directive the
>> response is correct again.
>>
>> Doing similar tests with CGI/PHP applications always show the correct
>> response without a response body. This only affects requests which
>> use mod_jk.
>>
>> So far I could reproduce this on SLES12 SP5 and SLES15 SP3 running
>> Apache httpd 2.4.51 and a self compile mod_jk 1.2.46 (same with the
>> included mod_jk 1.2.43) at work. At home the same happens on a stock
>> openSUSE Leap 15.3 with Apache httpd 2.4.51 and mod_jk 1.2.43 as well
>> as on Ubuntu 20.04 with httpd 2.4.41 and mod_jk 1.2.46.
>> I didn't try to compile the latest mod_jk version yet because I
>> didn't spot a relevant point in the changelog.
>>
>> Can anyone confirm this behaviour or point me to a configuration
>> directive i missed?
>>
--
Regards,
Stefan Mayr
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: mod_jk interference with ErrorDocument/Alias on HEAD request
Posted by Stefan Mayr <st...@mayr-stefan.de>.
Hello again,
a self-compiled mod_jk 1.2.48 shows the same issue.
Am 13.02.2022 um 18:37 schrieb Stefan Mayr:
> Hi,
>
> looking at the source code
> https://github.com/apache/tomcat-connectors/blob/main/native/apache-2.0/mod_jk.c#L2954#L2973
> I did some more testing:
>
> Variant 1: JkMount /demo/* ajp13_worker;use_server_errors=401
> Variant 2: JkMount /demo/* ajp13_worker
>
> ignoring what variant 2 changes for regular request: reading the source
> comment my understanding is, that for both variants a HEAD request (by
> definition must have an empty response body) should let Apache httpd
> handle the error code.
>
> But the return code for jk_handler looks different:
>
> Variant 1: s.http_response_status
> Variant 2: r->status
Although this looks different on the first glance it seems to be the same.
> The response only seems correct for variant 1 - which is configured to
> let Apache httpd handle all responses for status codes >= 401. For
> variant 2 mod_jk seems to handle the response itself - contrary to what
> the comment explains.
This leads to the next assumption, that whenever there is a special
handling for use_server_errors there should be something similar for the
case with an empty/non-existing response body.
There is
https://github.com/apache/tomcat-connectors/blob/main/native/common/jk_ajp_common.c#L1991#L1993
with no corresponding (pseudo) code like
if (!r->something_like_bodyct && r->http_response_status >=
JK_HTTP_BAD_REQUEST){
r->response_blocked = JK_TRUE;
}
Adding code like this (sorry, i could not find out how to determine if
there is a response body) fixes the issue with the wrong response body
for a HEAD request. But we miss the WWW-Authenticate header now.
Digging further we find
https://github.com/apache/tomcat-connectors/blob/main/native/apache-2.0/mod_jk.c#L331#L353
which has a special treatment for 401 HTTP Unauthorized. But again, only
for use_server_errors.
This should be fixable by extending the condition like this
if ((s->extension.use_server_error_pages &&
status >= s->extension.use_server_error_pages) ||
(!r->sent_bodyct && r->status >= HTTP_BAD_REQUEST)) {
But the WWW-Authenticate header is still missing. So i'm wrong, again.
Although it feels like i'm close.
> Am 12.02.2022 um 14:24 schrieb Stefan Mayr:
>> Hello Tomcat users,
>>
>> this week we were debugging a strange connection issue which I tracked
>> down to an interference between Apache httpd and mod_jk.
>>
>> For the full picture, the infrastructure setup contains
>>
>> 1. a Loadbalancer providing HTTPS, HTTP/1.1 and HTTP/2 for Clients.
>> 2. an Apache httpd 2.4 webserver (HTTP only) with mod_jk
>> 3. a Tomcat mit AJP-Connector
>>
>> We have an application doing many different HEAD requests against an
>> application running in the Tomcat server. The requests contain an
>> Authorization header for Basic authentication. Expected response is a
>> HTTP 200 OK or HTTP 401 if this particular user is not allowed to
>> access that ressource. Because this is a HEAD request there must not
>> be a response body according to RFC 2616.
>>
>> If there is a response body in the response to the HEAD request our
>> loadbalancer does strange things: aborts the connection if the clients
>> uses HTTP/2, SSL errors if using HTTP/1.1. But this is an issue on its
>> own which we might have to solve with the vendor.
>>
>> Now comes the point where I need your help. We have a httpd
>> configuration with mod_jk which generates these invalid response
>> bodies on HEAD requests. I have a gut feeling this could be a bug with
>> mod_jk.
>>
>> For demonstration purpose i created a minimal demo app which only is a
>> WEB-INF/web.xml
>> <?xml version="1.0" encoding="UTF-8"?>
>> <web-app xmlns="https://jakarta.ee/xml/ns/jakartaee"
>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>> xsi:schemaLocation="https://jakarta.ee/xml/ns/jakartaee
>>
>> https://jakarta.ee/xml/ns/jakartaee/web-app_5_0.xsd"
>> version="5.0">
>> <security-constraint>
>> <web-resource-collection>
>> <web-resource-name>Login</web-resource-name>
>> <url-pattern>/*</url-pattern>
>> </web-resource-collection>
>> <auth-constraint>
>> <role-name>manager</role-name>
>> </auth-constraint>
>> </security-constraint>
>> <security-role>
>> <role-name>manager</role-name>
>> </security-role>
>> <login-config>
>> <auth-method>BASIC</auth-method>
>> </login-config>
>> </web-app>
>>
>> Then I place a JkMount in my Apache httpd configuration (+ minimal
>> worker.properties):
>>
>> JkMount /demo/* ajp13_worker
>>
>> Testing this with curl works like expected:
>>
>> root@1ae8973f1b6b:~# curl -I -v localhost/demo/
>> * Trying 127.0.0.1:80...
>> * TCP_NODELAY set
>> * Connected to localhost (127.0.0.1) port 80 (#0)
>> > HEAD /demo/ HTTP/1.1
>> > Host: localhost
>> > User-Agent: curl/7.68.0
>> > Accept: */*
>> >
>> * Mark bundle as not supporting multiuse
>> < HTTP/1.1 401 401
>> HTTP/1.1 401 401
>> < Date: Sat, 12 Feb 2022 12:57:33 GMT
>> Date: Sat, 12 Feb 2022 12:57:33 GMT
>> < Server: Apache/2.4.41 (Ubuntu)
>> Server: Apache/2.4.41 (Ubuntu)
>> < Cache-Control: private
>> Cache-Control: private
>> < WWW-Authenticate: Basic realm="Authentication required"
>> WWW-Authenticate: Basic realm="Authentication required"
>> < Content-Language: en
>> Content-Language: en
>> < Content-Type: text/html;charset=utf-8
>> Content-Type: text/html;charset=utf-8
>>
>> <
>> * Connection #0 to host localhost left intact
>>
>> But our default setup always includes custom error pages:
>>
>> Alias /error/ "/usr/share/apache2/error/"
>> ErrorDocument 401 /error/HTTP_UNAUTHORIZED.html.var
>>
>> If both of those lines are added this results in a response body for
>> the HEAD request.
>>
>> root@1ae8973f1b6b:~# curl -I -v localhost/demo/
>> * Trying 127.0.0.1:80...
>> * TCP_NODELAY set
>> * Connected to localhost (127.0.0.1) port 80 (#0)
>> > HEAD /demo/ HTTP/1.1
>> > Host: localhost
>> > User-Agent: curl/7.68.0
>> > Accept: */*
>> >
>> * Mark bundle as not supporting multiuse
>> < HTTP/1.1 401 401
>> HTTP/1.1 401 401
>> < Date: Sat, 12 Feb 2022 12:56:27 GMT
>> Date: Sat, 12 Feb 2022 12:56:27 GMT
>> < Server: Apache/2.4.41 (Ubuntu)
>> Server: Apache/2.4.41 (Ubuntu)
>> < Cache-Control: private
>> Cache-Control: private
>> < WWW-Authenticate: Basic realm="Authentication required"
>> WWW-Authenticate: Basic realm="Authentication required"
>> < Content-Language: en
>> Content-Language: en
>> < Content-Type: text/html;charset=utf-8
>> Content-Type: text/html;charset=utf-8
>>
>> <
>> * Excess found: excess = 589 url = /demo/ (zero-length body)
>> * Connection #0 to host localhost left intact
>>
>> Checking with tcpdump on port 8009 we see the expected response
>> without a body from the Tomcat AJP connector. The tcpdump von port 80
>> reveals httpd is adding the configured ErrorDocument as response body.
>>
>> If we comment out either the Alias or ErrorDocument directive the
>> response is correct again.
>>
>> Doing similar tests with CGI/PHP applications always show the correct
>> response without a response body. This only affects requests which use
>> mod_jk.
>>
>> So far I could reproduce this on SLES12 SP5 and SLES15 SP3 running
>> Apache httpd 2.4.51 and a self compile mod_jk 1.2.46 (same with the
>> included mod_jk 1.2.43) at work. At home the same happens on a stock
>> openSUSE Leap 15.3 with Apache httpd 2.4.51 and mod_jk 1.2.43 as well
>> as on Ubuntu 20.04 with httpd 2.4.41 and mod_jk 1.2.46.
>> I didn't try to compile the latest mod_jk version yet because I didn't
>> spot a relevant point in the changelog.
>>
>> Can anyone confirm this behaviour or point me to a configuration
>> directive i missed?
>>
--
Regards,
Stefan Mayr
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
AW: mod_jk interference with ErrorDocument/Alias on HEAD request
Posted by "Thomas Hoffmann (Speed4Trade GmbH)" <Th...@speed4trade.com.INVALID>.
Hello,
the spec https://datatracker.ietf.org/doc/html/rfc7231#page-25 says in chapter 4.3.2:
" The HEAD method is identical to GET except that the server MUST NOT
send a message body in the response (i.e., the response terminates at
the end of the header section)."
Greetings,
Thomas
-----Ursprüngliche Nachricht-----
Von: Stefan Mayr <st...@mayr-stefan.de>
Gesendet: Sonntag, 13. Februar 2022 18:37
An: users@tomcat.apache.org
Betreff: Re: mod_jk interference with ErrorDocument/Alias on HEAD request
Hi,
looking at the source code
https://github.com/apache/tomcat-connectors/blob/main/native/apache-2.0/mod_jk.c#L2954#L2973
I did some more testing:
Variant 1: JkMount /demo/* ajp13_worker;use_server_errors=401
Variant 2: JkMount /demo/* ajp13_worker
ignoring what variant 2 changes for regular request: reading the source comment my understanding is, that for both variants a HEAD request (by definition must have an empty response body) should let Apache httpd handle the error code.
But the return code for jk_handler looks different:
Variant 1: s.http_response_status
Variant 2: r->status
The response only seems correct for variant 1 - which is configured to let Apache httpd handle all responses for status codes >= 401. For variant 2 mod_jk seems to handle the response itself - contrary to what the comment explains.
Am 12.02.2022 um 14:24 schrieb Stefan Mayr:
> Hello Tomcat users,
>
> this week we were debugging a strange connection issue which I tracked
> down to an interference between Apache httpd and mod_jk.
>
> For the full picture, the infrastructure setup contains
>
> 1. a Loadbalancer providing HTTPS, HTTP/1.1 and HTTP/2 for Clients.
> 2. an Apache httpd 2.4 webserver (HTTP only) with mod_jk 3. a Tomcat
> mit AJP-Connector
>
> We have an application doing many different HEAD requests against an
> application running in the Tomcat server. The requests contain an
> Authorization header for Basic authentication. Expected response is a
> HTTP 200 OK or HTTP 401 if this particular user is not allowed to
> access that ressource. Because this is a HEAD request there must not
> be a response body according to RFC 2616.
>
> If there is a response body in the response to the HEAD request our
> loadbalancer does strange things: aborts the connection if the clients
> uses HTTP/2, SSL errors if using HTTP/1.1. But this is an issue on its
> own which we might have to solve with the vendor.
>
> Now comes the point where I need your help. We have a httpd
> configuration with mod_jk which generates these invalid response
> bodies on HEAD requests. I have a gut feeling this could be a bug with mod_jk.
>
> For demonstration purpose i created a minimal demo app which only is a
> WEB-INF/web.xml <?xml version="1.0" encoding="UTF-8"?> <web-app
> xmlns="https://jakarta.ee/xml/ns/jakartaee"
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> xsi:schemaLocation="https://jakarta.ee/xml/ns/jakartaee
> https://jakarta.ee/xml/ns/jakartaee/web-app_5_0.xsd"
> version="5.0">
> <security-constraint>
> <web-resource-collection>
> <web-resource-name>Login</web-resource-name>
> <url-pattern>/*</url-pattern>
> </web-resource-collection>
> <auth-constraint>
> <role-name>manager</role-name>
> </auth-constraint>
> </security-constraint>
> <security-role>
> <role-name>manager</role-name>
> </security-role>
> <login-config>
> <auth-method>BASIC</auth-method>
> </login-config>
> </web-app>
>
> Then I place a JkMount in my Apache httpd configuration (+ minimal
> worker.properties):
>
> JkMount /demo/* ajp13_worker
>
> Testing this with curl works like expected:
>
> root@1ae8973f1b6b:~# curl -I -v localhost/demo/
> * Trying 127.0.0.1:80...
> * TCP_NODELAY set
> * Connected to localhost (127.0.0.1) port 80 (#0) > HEAD /demo/
> HTTP/1.1 > Host: localhost > User-Agent: curl/7.68.0 > Accept: */*
> >
> * Mark bundle as not supporting multiuse < HTTP/1.1 401 401
> HTTP/1.1 401 401
> < Date: Sat, 12 Feb 2022 12:57:33 GMT
> Date: Sat, 12 Feb 2022 12:57:33 GMT
> < Server: Apache/2.4.41 (Ubuntu)
> Server: Apache/2.4.41 (Ubuntu)
> < Cache-Control: private
> Cache-Control: private
> < WWW-Authenticate: Basic realm="Authentication required"
> WWW-Authenticate: Basic realm="Authentication required"
> < Content-Language: en
> Content-Language: en
> < Content-Type: text/html;charset=utf-8
> Content-Type: text/html;charset=utf-8
>
> <
> * Connection #0 to host localhost left intact
>
> But our default setup always includes custom error pages:
>
> Alias /error/ "/usr/share/apache2/error/"
> ErrorDocument 401 /error/HTTP_UNAUTHORIZED.html.var
>
> If both of those lines are added this results in a response body for
> the HEAD request.
>
> root@1ae8973f1b6b:~# curl -I -v localhost/demo/
> * Trying 127.0.0.1:80...
> * TCP_NODELAY set
> * Connected to localhost (127.0.0.1) port 80 (#0) > HEAD /demo/
> HTTP/1.1 > Host: localhost > User-Agent: curl/7.68.0 > Accept: */*
> >
> * Mark bundle as not supporting multiuse < HTTP/1.1 401 401
> HTTP/1.1 401 401
> < Date: Sat, 12 Feb 2022 12:56:27 GMT
> Date: Sat, 12 Feb 2022 12:56:27 GMT
> < Server: Apache/2.4.41 (Ubuntu)
> Server: Apache/2.4.41 (Ubuntu)
> < Cache-Control: private
> Cache-Control: private
> < WWW-Authenticate: Basic realm="Authentication required"
> WWW-Authenticate: Basic realm="Authentication required"
> < Content-Language: en
> Content-Language: en
> < Content-Type: text/html;charset=utf-8
> Content-Type: text/html;charset=utf-8
>
> <
> * Excess found: excess = 589 url = /demo/ (zero-length body)
> * Connection #0 to host localhost left intact
>
> Checking with tcpdump on port 8009 we see the expected response
> without a body from the Tomcat AJP connector. The tcpdump von port 80
> reveals httpd is adding the configured ErrorDocument as response body.
>
> If we comment out either the Alias or ErrorDocument directive the
> response is correct again.
>
> Doing similar tests with CGI/PHP applications always show the correct
> response without a response body. This only affects requests which use
> mod_jk.
>
> So far I could reproduce this on SLES12 SP5 and SLES15 SP3 running
> Apache httpd 2.4.51 and a self compile mod_jk 1.2.46 (same with the
> included mod_jk 1.2.43) at work. At home the same happens on a stock
> openSUSE Leap 15.3 with Apache httpd 2.4.51 and mod_jk 1.2.43 as well
> as on Ubuntu 20.04 with httpd 2.4.41 and mod_jk 1.2.46.
> I didn't try to compile the latest mod_jk version yet because I didn't
> spot a relevant point in the changelog.
>
> Can anyone confirm this behaviour or point me to a configuration
> directive i missed?
>
> Thank you,
>
>
> Stefan Mayr
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
Regards,
Stefan Mayr
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: mod_jk interference with ErrorDocument/Alias on HEAD request
Posted by Stefan Mayr <st...@mayr-stefan.de>.
Hi,
looking at the source code
https://github.com/apache/tomcat-connectors/blob/main/native/apache-2.0/mod_jk.c#L2954#L2973
I did some more testing:
Variant 1: JkMount /demo/* ajp13_worker;use_server_errors=401
Variant 2: JkMount /demo/* ajp13_worker
ignoring what variant 2 changes for regular request: reading the source
comment my understanding is, that for both variants a HEAD request (by
definition must have an empty response body) should let Apache httpd
handle the error code.
But the return code for jk_handler looks different:
Variant 1: s.http_response_status
Variant 2: r->status
The response only seems correct for variant 1 - which is configured to
let Apache httpd handle all responses for status codes >= 401. For
variant 2 mod_jk seems to handle the response itself - contrary to what
the comment explains.
Am 12.02.2022 um 14:24 schrieb Stefan Mayr:
> Hello Tomcat users,
>
> this week we were debugging a strange connection issue which I tracked
> down to an interference between Apache httpd and mod_jk.
>
> For the full picture, the infrastructure setup contains
>
> 1. a Loadbalancer providing HTTPS, HTTP/1.1 and HTTP/2 for Clients.
> 2. an Apache httpd 2.4 webserver (HTTP only) with mod_jk
> 3. a Tomcat mit AJP-Connector
>
> We have an application doing many different HEAD requests against an
> application running in the Tomcat server. The requests contain an
> Authorization header for Basic authentication. Expected response is a
> HTTP 200 OK or HTTP 401 if this particular user is not allowed to access
> that ressource. Because this is a HEAD request there must not be a
> response body according to RFC 2616.
>
> If there is a response body in the response to the HEAD request our
> loadbalancer does strange things: aborts the connection if the clients
> uses HTTP/2, SSL errors if using HTTP/1.1. But this is an issue on its
> own which we might have to solve with the vendor.
>
> Now comes the point where I need your help. We have a httpd
> configuration with mod_jk which generates these invalid response bodies
> on HEAD requests. I have a gut feeling this could be a bug with mod_jk.
>
> For demonstration purpose i created a minimal demo app which only is a
> WEB-INF/web.xml
> <?xml version="1.0" encoding="UTF-8"?>
> <web-app xmlns="https://jakarta.ee/xml/ns/jakartaee"
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> xsi:schemaLocation="https://jakarta.ee/xml/ns/jakartaee
> https://jakarta.ee/xml/ns/jakartaee/web-app_5_0.xsd"
> version="5.0">
> <security-constraint>
> <web-resource-collection>
> <web-resource-name>Login</web-resource-name>
> <url-pattern>/*</url-pattern>
> </web-resource-collection>
> <auth-constraint>
> <role-name>manager</role-name>
> </auth-constraint>
> </security-constraint>
> <security-role>
> <role-name>manager</role-name>
> </security-role>
> <login-config>
> <auth-method>BASIC</auth-method>
> </login-config>
> </web-app>
>
> Then I place a JkMount in my Apache httpd configuration (+ minimal
> worker.properties):
>
> JkMount /demo/* ajp13_worker
>
> Testing this with curl works like expected:
>
> root@1ae8973f1b6b:~# curl -I -v localhost/demo/
> * Trying 127.0.0.1:80...
> * TCP_NODELAY set
> * Connected to localhost (127.0.0.1) port 80 (#0)
> > HEAD /demo/ HTTP/1.1
> > Host: localhost
> > User-Agent: curl/7.68.0
> > Accept: */*
> >
> * Mark bundle as not supporting multiuse
> < HTTP/1.1 401 401
> HTTP/1.1 401 401
> < Date: Sat, 12 Feb 2022 12:57:33 GMT
> Date: Sat, 12 Feb 2022 12:57:33 GMT
> < Server: Apache/2.4.41 (Ubuntu)
> Server: Apache/2.4.41 (Ubuntu)
> < Cache-Control: private
> Cache-Control: private
> < WWW-Authenticate: Basic realm="Authentication required"
> WWW-Authenticate: Basic realm="Authentication required"
> < Content-Language: en
> Content-Language: en
> < Content-Type: text/html;charset=utf-8
> Content-Type: text/html;charset=utf-8
>
> <
> * Connection #0 to host localhost left intact
>
> But our default setup always includes custom error pages:
>
> Alias /error/ "/usr/share/apache2/error/"
> ErrorDocument 401 /error/HTTP_UNAUTHORIZED.html.var
>
> If both of those lines are added this results in a response body for the
> HEAD request.
>
> root@1ae8973f1b6b:~# curl -I -v localhost/demo/
> * Trying 127.0.0.1:80...
> * TCP_NODELAY set
> * Connected to localhost (127.0.0.1) port 80 (#0)
> > HEAD /demo/ HTTP/1.1
> > Host: localhost
> > User-Agent: curl/7.68.0
> > Accept: */*
> >
> * Mark bundle as not supporting multiuse
> < HTTP/1.1 401 401
> HTTP/1.1 401 401
> < Date: Sat, 12 Feb 2022 12:56:27 GMT
> Date: Sat, 12 Feb 2022 12:56:27 GMT
> < Server: Apache/2.4.41 (Ubuntu)
> Server: Apache/2.4.41 (Ubuntu)
> < Cache-Control: private
> Cache-Control: private
> < WWW-Authenticate: Basic realm="Authentication required"
> WWW-Authenticate: Basic realm="Authentication required"
> < Content-Language: en
> Content-Language: en
> < Content-Type: text/html;charset=utf-8
> Content-Type: text/html;charset=utf-8
>
> <
> * Excess found: excess = 589 url = /demo/ (zero-length body)
> * Connection #0 to host localhost left intact
>
> Checking with tcpdump on port 8009 we see the expected response without
> a body from the Tomcat AJP connector. The tcpdump von port 80 reveals
> httpd is adding the configured ErrorDocument as response body.
>
> If we comment out either the Alias or ErrorDocument directive the
> response is correct again.
>
> Doing similar tests with CGI/PHP applications always show the correct
> response without a response body. This only affects requests which use
> mod_jk.
>
> So far I could reproduce this on SLES12 SP5 and SLES15 SP3 running
> Apache httpd 2.4.51 and a self compile mod_jk 1.2.46 (same with the
> included mod_jk 1.2.43) at work. At home the same happens on a stock
> openSUSE Leap 15.3 with Apache httpd 2.4.51 and mod_jk 1.2.43 as well as
> on Ubuntu 20.04 with httpd 2.4.41 and mod_jk 1.2.46.
> I didn't try to compile the latest mod_jk version yet because I didn't
> spot a relevant point in the changelog.
>
> Can anyone confirm this behaviour or point me to a configuration
> directive i missed?
>
> Thank you,
>
>
> Stefan Mayr
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
Regards,
Stefan Mayr
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org