You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Dennis Rech <dr...@wikon.de> on 2020/01/06 12:09:31 UTC
Ignore duplicate HTTP headers in Tomcat 8.5.50-0+deb9u1
Hi and happy new year,
we have an application where HTTP clients have a kind of unclean way of
submitting HTTP POST requests to our tomcat server for data upload: The
|POST| and |Host: xxx| part appears twice in the request.
Until now this didn't cause any problems with tomcat, but since the
latest release, Tomcat refuses to accept this message and returns a 400
bad request immediately.
Unfortunately we'll not be able to change the client-side code. Is there
any way to tell the tomcat connector "ignore duplicate headers" or so to
make it work again? I guess the rewrite filters for tomcat won't help as
tomcat probably discards the incoming message before handing it over to
rewrite.
Example request:
|POST /data/upload/test HTTP/1.1 Host: www.myhost.de:8180 POST
/data/upload/test HTTP/1.1 Host: www.myhost.de:8180 [...rest of the
request is ok...] |
I wonder if there is a parameter for the |Connector| part in server.xml
or so to workaround this problem and restore the old behaviour without
downgrading.
Greetings,
Dennis
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Ignore duplicate HTTP headers in Tomcat 8.5.50-0+deb9u1
Posted by logo <lo...@kreuser.name>.
Dennis,
>
> Am 07.01.2020 um 08:10 schrieb Dennis Rech <dr...@wikon.de>:
>
> Hi Christopher,
>
>> Am 06.01.20 um 17:39 schrieb Christopher Schultz:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA256
>>
>> Dennia,
>>
>>> On 1/6/20 07:09, Dennis Rech wrote:
>>> we have an application where HTTP clients have a kind of unclean
>>> way of submitting HTTP POST requests to our tomcat server for data
>>> upload: The |POST| and |Host: xxx| part appears twice in the
>>> request.
>> Yuck. You mean like this?
>>
>> POST /foo HTTP/1.1
>> POST /foo HTTP/1.1
>> Host: foo.com
>> Host: foo.com
>> Content-Type: application/x-www-url-encoded
>> Content-Length: 13
>>
>> q=Hello World
>>
>> ?
>
> No, rather like that:
>
> POST /foo HTTP/1.1
> Host: foo.com
> POST /foo HTTP/1.1
> Host: foo.com
> Content-[stuff] [...]
>
It could be worse. But at least the first two lines are protocol
compliant.
As Chris mentioned you may add a reverse proxy like nginx or traefik in
front of it and strip the lines - the POST line not a „real“ header
though - before passing the request to tomcat.
Then if the first two lines are compliant, I wonder if a Valve on tomcat
level could intercept the request and deal with the bad lines before
reaching the app layer.
Peter
>>
>>> Until now this didn't cause any problems with tomcat, but since
>>> the latest release, Tomcat refuses to accept this message and
>>> returns a 400 bad request immediately.
>> Having two "host" headers should be okay. But repeating the request
>> line is a clear violation of the HTTP spec that will be difficult to
>> get over. I can't believe Tomcat ever allowed that, though it may have
>> done so.
> I read in the changelog that since Tomcat 8.5.22 it will also reply
> with Bad request 400 if there are two Host fields in the header. But I
> guess the double "POST" is even worse.
>>
>>> Unfortunately we'll not be able to change the client-side code. Is
>>> there any way to tell the tomcat connector "ignore duplicate
>>> headers" or so to make it work again? I guess the rewrite filters
>>> for tomcat won't help as tomcat probably discards the incoming
>>> message before handing it over to rewrite.
>> Tomcat is responsible for reading the request line and routing the
>> request to an application. If the request is broken badly enough, it
>> won't be able to route.
>>
>> Headers are parsed as a part of that, and:
>>
>> POST /foo HTTP/1.1
>>
>> is not a valid header for at least two reasons:
>>
>> 1. There is no : character (required, even when the header has no
>> value)
>> 2. There are spaces in the "name" (the name is everything before colon
>> )
> Well, "POST"... is the actual request followed by the HTTP headers.
> POST is not part of the actual header. Maybe I haven't pointed that
> out.
>>
>>> Example request:
>>>
>>> |POST /data/upload/test HTTP/1.1 Host: www.myhost.de:8180 POST
>>> /data/upload/test HTTP/1.1 Host: www.myhost.de:8180 [...rest of
>>> the request is ok...] |
>> This got word-wrapped. Was this?
>
> Yes I copied it from a formatted document, the pipes probably indicate
> that this text was preformatted in the original document, sorry. Also
> the newlines are missing.
>
> POST /data/upload/test HTTP/1.1
> Host: www.myhost.de:8180
> POST/data/upload/test HTTP/1.1
> Host: www.myhost.de:8180
> [...here comes the remaining header with Content-Length etc followed by
> the body...]
>
>>
>>> POST /data/upload/test HTTP/1.1 Host: www.myhost.de:8180 POST
>>> /data/upload/test HTTP/1.1 Host: www.myhost.de:8180 [...rest of the
>>> request is ok...]
>> ?
>>
>> Yikes. What kind of client is this?
> It's a remote unit transmitting data to a server using POST file
> uploads with - obviously - a little bug in the firmware that builds the
> HTTP request manually as there is no curl library for the unit etc.
> which can be used to generate the requests.
>>
>>> I wonder if there is a parameter for the |Connector| part in
>>> server.xml or so to workaround this problem and restore the old
>>> behaviour without downgrading.
>> The good news is that the second POST could theoretically be
>> considered to be a "broken" header and ignored. But Tomcat has been
>> getting progressively more strict about what it will accept. There are
>> all kinds of nasty ways to use malformed messages like this to confuse
>> environments where e.g. a reverse-proxy and the origin server behave
>> differently when they see requests like those above. It's better to
>> just fail and fix the software. Why can't you fix the clients? Is this
>> another case of internet-of-things garbage that can't practically be
>> repaired?
>
> Something like that. The devices could be updated in theory but
> probably not over-the-air and many of them are already deployed
> somewhere in the "wild" so we don't have physical access to them
> anymore. Unfortunately we did not notice that in the past as tomcat
> always accepted these requests until the latest update for debian came
> out.
>
> I totally agree that the best solution would be to let those devices
> send proper HTTP protocol but I guess we'll have to find a workaround
> on server-side.
>
>>
>> - -chris
>> -----BEGIN PGP SIGNATURE-----
>> Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/
>>
>> iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl4TYrsACgkQHPApP6U8
>> pFjNyRAAnebbKRBT99Zgk8vfnW4gUOGXYlgX8Xk2W8bUI7UPfqpphS6+t6Lwjti1
>> SBnS2qKuUXELMI7fca5Iq2NhKD9cCxZhuWrRQ6+jSZESaT6EEGeKYFaJ3Jtn5Dv6
>> ll6JKASuNTZtaDo+df0xYnDUvk6kT99aM3GY4FwYNLl/FIxv58D6YbUXN0TYLJQy
>> 0IrnalQEQO8b8SiZcOPPAeA8ODZJQ37egAe3kP3xyWQPMl2u/966c0b711qZJ8JY
>> D9JaR0fnVTZMHCHQmK9xLQKqra4T7uOv0UslQVheq47+SuXJea+qz1AmSJf3CU57
>> bL4zD7XVKkcC/bsOu1uWUSaKjN0KRnIqwxwo2trpPyoMpKJBGY26GdekpM/jeLdU
>> eyh4I8f8fJkJ7GerLZrZSGxKIlTs86fzM96wr/P4pgSclBjgCO3RMf9phl7URmgp
>> rIqT/xjwjIyLWqASWhXUyT7F5EHbZLSvPRbvklAib2pwQU2taaeT4cZaWGB4wfWw
>> +CXQgGj7E4bWmTeq4LsG3skJEVcYcVN/AHSsgWQ5MFsjKx5pxJjq++P+ZooTqQOr
>> GNexi9KVFilovrf1iStoP4OgdPcf2o1l0PXCpbKf5i2woJgRpFS/oSlOno0KQaol
>> hvddVhzgL8PYrcWywdGEIbzI/9eQm54nqOAMRlpOCTOaDPzbeLc=
>> =vIL0
>> -----END PGP SIGNATURE-----
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Ignore duplicate HTTP headers in Tomcat 8.5.50-0+deb9u1
Posted by Dennis Rech <dr...@wikon.de>.
Dear Mark,
thanks a lot for your effort and your feedback.
Am 07.01.20 um 10:36 schrieb Mark Thomas:
> On 07/01/2020 07:10, Dennis Rech wrote:
>> POST /foo HTTP/1.1
>> Host: foo.com
>> POST /foo HTTP/1.1
>> Host: foo.com
>> Content-[stuff] [...]
> First two lines are OK.
>
> The third line is going to be treated as an HTTP header. It is invalid
> and Tomcat will reject it with a 400 response but you can tell Tomcat to
> just ignore the invalid header with rejectIllegalHeaderName="false" on
> the Connector.
I already tried the connector option in server.xml and also saw in the
documentation the
rejectIllegalHeaderName
option should be on "false" even as default.
> The problem is going to be the second Host header.
The second "Host" part is still a problem, you're right.
We have just tried putting nginx as a proxy before the tomcat8 instance
and it has intercepted the malformed request fine at least it has not
rejected the request with HTTP response code 400.
>
> RFC 7230 states:
>
> <quote>
> A server MUST respond with a 400 (Bad Request) status code to any
> HTTP/1.1 request message that lacks a Host header field and to any
> request message that contains more than one Host header field or a
> Host header field with an invalid field-value.
> </quote>
>
> Any spec compliant server is almost certainly going to reject that
> request. I guess a server might provide a hook for request modification
> prior to rejection to allow the "fixing" of known invalid requests but
> I'm not aware of any that do - at least not without going down the
> writing a custom module route.
>
> If we made Http11Processor.prepareRequest() protected then it would be
> fairly simple to write a custom Processor that:
> - extended Http11Processor
> - overrode prepareRequest() to
> a) remove the duplicate Host header
> b) call super.prepareRequest()
>
> I could provide one for you if you weren't comfortable doing that
> yourself). However, even if we made the change now (which I'm happy to
> do if you think it would be useful) it will take a while to filter
> through to the Debian distribution.
>
> There are several variations on this theme. One could write a custom
> Processor for 8.5.50 that did the same thing - it would just be rather
> more involved as one would have to copy rather more code from
> Http11Processor.
We will see if the nginx workaround is sufficient :-). Otherwise we will
take into consideration your suggestion.
Thanks again.
Dennis
>
> Mark
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Ignore duplicate HTTP headers in Tomcat 8.5.50-0+deb9u1
Posted by Peter Kreuser <lo...@kreuser.name>.
Chris (and Mark),
> Am 07.01.2020 um 17:22 schrieb Christopher Schultz <ch...@christopherschultz.net>:
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> Mark,
>
>> On 1/7/20 4:36 AM, Mark Thomas wrote:
>>> On 07/01/2020 07:10, Dennis Rech wrote:
>>> POST /foo HTTP/1.1 Host: foo.com POST /foo HTTP/1.1 Host:
>>> foo.com Content-[stuff] [...]
>>
>> First two lines are OK.
>>
>> The third line is going to be treated as an HTTP header. It is
>> invalid and Tomcat will reject it with a 400 response but you can
>> tell Tomcat to just ignore the invalid header with
>> rejectIllegalHeaderName="false" on the Connector.
>>
>> The problem is going to be the second Host header.
>>
>> RFC 7230 states:
>>
>> <quote> A server MUST respond with a 400 (Bad Request) status code
>> to any HTTP/1.1 request message that lacks a Host header field and
>> to any request message that contains more than one Host header
>> field or a Host header field with an invalid field-value. </quote>
>>
>> Any spec compliant server is almost certainly going to reject that
>> request. I guess a server might provide a hook for request
>> modification prior to rejection to allow the "fixing" of known
>> invalid requests but I'm not aware of any that do - at least not
>> without going down the writing a custom module route.
>>
>> If we made Http11Processor.prepareRequest() protected then it would
>> be fairly simple to write a custom Processor that: - extended
>> Http11Processor - overrode prepareRequest() to a) remove the
>> duplicate Host header b) call super.prepareRequest()
>>
>> I could provide one for you if you weren't comfortable doing that
>> yourself). However, even if we made the change now (which I'm happy
>> to do if you think it would be useful) it will take a while to
>> filter through to the Debian distribution.
>>
>> There are several variations on this theme. One could write a
>> custom Processor for 8.5.50 that did the same thing - it would just
>> be rather more involved as one would have to copy rather more code
>> from Http11Processor.
>
> Since the Host header seems to be special in this regard (i.e. there
> is no prohibition against multiple Accept headers), might we be
> willing to interpret the spec in a slightly less strict manner?
>
> "
> A server MUST respond with a 400 (Bad Request) status code to any
> HTTP/1.1 request message that lacks a Host header field and to any
> request message that contains more than one Host header field [[WITH A
> CONFLICTING VALUE]]] or a Host header field with an invalid field-value.
> "
That would be a good idea - maybe only in conjunction with setting rejectIllegalHeaderName=false
If that makes the implementation easier???
>
> So a request with:
>
> Host: foo.bar.com
> Host: foo.bar.com
>
> Would be okay, while:
>
> Host: foo.bar.com
> Host: bar.foo.com
>
> Would return a 400 response?
>
That would be a messed up request and worth a 400!
Peter
> - -chris
> -----BEGIN PGP SIGNATURE-----
> Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/
>
> iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl4UsEEACgkQHPApP6U8
> pFjprxAAi60mVwH+LHo2HSCl+hwIhIyG2B9Dg2LjIJ8JPMA/WxaiDOOJCS+yMTbV
> rVfdPrlT0B6Zd8ceTjz4ooZa79SwPFfCiFM97q1H/JwwsqVxaBEFEx6PgvnJzUUF
> ZuJJEtQHijQgZo0gXv2plkqHTBrG5NMPNqQYEJ8aZqdjtSvtNkP2E03agVC/8SqW
> mNyERNFcyOP3hUlNHSghPXl81ckSabqa83rLrwFCZQGJ2U71EnnietYxXT5Dz6Kx
> W03z8HY2mClTETmZB/WvkCmG0F1AXQ8Xr2E4fJ2+meyNHgTZ2XjfYsKtZNKTQmiC
> zlDgweuXuQ1r6DorLB4MUCm7HMffeDTwKEHBaYkIt7reHN8yGfT8sq1F8A0ZDKHi
> y9Ugt0KwePPOGFK8mfST7lBWojPJL1wbyBVAYh+FL5f1hMScOdHRxbU9uz2p9NSB
> RMubUWNCD1p8+sI8bLjQ//vU/iCLcWg7RStr/FSfXZEqjJv6EZ4OaNafahTcxvey
> 37Qz/eVTJQGeYa0+1rBvttVZJB6xrJwcscC3dgskTJ8VXJuAnwK0WdmMRzD7XLos
> HP13SOoLXUgek07XH61OPq5dnbpUwq996GqpSLldLUJlCnbMi1vxkAmGe006zVXH
> GWPoV1d4r7p0JjkyBlGQYUwiltuDFyNOx9uRS5FTaapaarhY6G0=
> =Z571
> -----END PGP SIGNATURE-----
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Ignore duplicate HTTP headers in Tomcat 8.5.50-0+deb9u1
Posted by Peter Kreuser <lo...@kreuser.name>.
Mark,
maybe this getting offtopic.
> Am 07.01.2020 um 18:58 schrieb Mark Thomas <ma...@apache.org>:
>
> On 07/01/2020 16:22, Christopher Schultz wrote:
>
> <snip/>
>
>> Since the Host header seems to be special in this regard (i.e. there
>> is no prohibition against multiple Accept headers), might we be
>> willing to interpret the spec in a slightly less strict manner?
>>
>> "
>> A server MUST respond with a 400 (Bad Request) status code to any
>> HTTP/1.1 request message that lacks a Host header field and to any
>> request message that contains more than one Host header field [[WITH A
>> CONFLICTING VALUE]]] or a Host header field with an invalid field-value.
>> "
>>
>> So a request with:
>>
>> Host: foo.bar.com
>> Host: foo.bar.com
>>
>> Would be okay, while:
>>
>> Host: foo.bar.com
>> Host: bar.foo.com
>>
>> Would return a 400 response?
>
> Not sure.
>
> The usual concern with multiple headers is that a reverse proxy uses
> one, the back-end uses another and you get unexpected behaviour that may
> result in some sort of information leak - i.e. a vulnerability. That
> shouldn't apply here since the values are the same.
>
> Experience suggests that being more relaxed about parsing an HTTP
> request that the specification requires is likely to result - at some
> point in the future - with a vulnerability report.
>
> In this instance I can image some other server somehow merging the two
> header values and - essentially - treating it as a different value to
> Tomcat. That could lead to a similar information disclosure as above.
>
I didn’t think that far! However, if they are the same and tomcat would also have to respond to the given host, that would be extremely unlikely...
> The usual counter argument is that there is no vulnerability if the
> other server is spec compliant. But the same is true if Tomcat is spec
> compliant.
>
> I certainly wouldn't support this behaviour by default.
>
Agreed.
> Making the behaviour configurable is possible so it could be enabled
> when necessary and safe to do so (i.e. clients connecting directly to
> Tomcat). That then gets you to the question how complex would this be to
> implement and is that complexity justified by the benefit it brings?
>
Just thinking how to handle “n” Host headers at various locations in the request... 8-0
> At this point, I'm not sure.
>
> So far we are looking at a feature required by a single user to handle
> clearly non-spec compliant clients. I lean more towards a custom
> protocol / processor implementation when a single user is affected.
>
That’s true.
Could this be a documented “extra”? Like a drop-in Jar?
Peter
> Mark
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Ignore duplicate HTTP headers in Tomcat 8.5.50-0+deb9u1
Posted by Mark Thomas <ma...@apache.org>.
On 07/01/2020 16:22, Christopher Schultz wrote:
<snip/>
> Since the Host header seems to be special in this regard (i.e. there
> is no prohibition against multiple Accept headers), might we be
> willing to interpret the spec in a slightly less strict manner?
>
> "
> A server MUST respond with a 400 (Bad Request) status code to any
> HTTP/1.1 request message that lacks a Host header field and to any
> request message that contains more than one Host header field [[WITH A
> CONFLICTING VALUE]]] or a Host header field with an invalid field-value.
> "
>
> So a request with:
>
> Host: foo.bar.com
> Host: foo.bar.com
>
> Would be okay, while:
>
> Host: foo.bar.com
> Host: bar.foo.com
>
> Would return a 400 response?
Not sure.
The usual concern with multiple headers is that a reverse proxy uses
one, the back-end uses another and you get unexpected behaviour that may
result in some sort of information leak - i.e. a vulnerability. That
shouldn't apply here since the values are the same.
Experience suggests that being more relaxed about parsing an HTTP
request that the specification requires is likely to result - at some
point in the future - with a vulnerability report.
In this instance I can image some other server somehow merging the two
header values and - essentially - treating it as a different value to
Tomcat. That could lead to a similar information disclosure as above.
The usual counter argument is that there is no vulnerability if the
other server is spec compliant. But the same is true if Tomcat is spec
compliant.
I certainly wouldn't support this behaviour by default.
Making the behaviour configurable is possible so it could be enabled
when necessary and safe to do so (i.e. clients connecting directly to
Tomcat). That then gets you to the question how complex would this be to
implement and is that complexity justified by the benefit it brings?
At this point, I'm not sure.
So far we are looking at a feature required by a single user to handle
clearly non-spec compliant clients. I lean more towards a custom
protocol / processor implementation when a single user is affected.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Ignore duplicate HTTP headers in Tomcat 8.5.50-0+deb9u1
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Mark,
On 1/7/20 4:36 AM, Mark Thomas wrote:
> On 07/01/2020 07:10, Dennis Rech wrote:
>> POST /foo HTTP/1.1 Host: foo.com POST /foo HTTP/1.1 Host:
>> foo.com Content-[stuff] [...]
>
> First two lines are OK.
>
> The third line is going to be treated as an HTTP header. It is
> invalid and Tomcat will reject it with a 400 response but you can
> tell Tomcat to just ignore the invalid header with
> rejectIllegalHeaderName="false" on the Connector.
>
> The problem is going to be the second Host header.
>
> RFC 7230 states:
>
> <quote> A server MUST respond with a 400 (Bad Request) status code
> to any HTTP/1.1 request message that lacks a Host header field and
> to any request message that contains more than one Host header
> field or a Host header field with an invalid field-value. </quote>
>
> Any spec compliant server is almost certainly going to reject that
> request. I guess a server might provide a hook for request
> modification prior to rejection to allow the "fixing" of known
> invalid requests but I'm not aware of any that do - at least not
> without going down the writing a custom module route.
>
> If we made Http11Processor.prepareRequest() protected then it would
> be fairly simple to write a custom Processor that: - extended
> Http11Processor - overrode prepareRequest() to a) remove the
> duplicate Host header b) call super.prepareRequest()
>
> I could provide one for you if you weren't comfortable doing that
> yourself). However, even if we made the change now (which I'm happy
> to do if you think it would be useful) it will take a while to
> filter through to the Debian distribution.
>
> There are several variations on this theme. One could write a
> custom Processor for 8.5.50 that did the same thing - it would just
> be rather more involved as one would have to copy rather more code
> from Http11Processor.
Since the Host header seems to be special in this regard (i.e. there
is no prohibition against multiple Accept headers), might we be
willing to interpret the spec in a slightly less strict manner?
"
A server MUST respond with a 400 (Bad Request) status code to any
HTTP/1.1 request message that lacks a Host header field and to any
request message that contains more than one Host header field [[WITH A
CONFLICTING VALUE]]] or a Host header field with an invalid field-value.
"
So a request with:
Host: foo.bar.com
Host: foo.bar.com
Would be okay, while:
Host: foo.bar.com
Host: bar.foo.com
Would return a 400 response?
- -chris
-----BEGIN PGP SIGNATURE-----
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/
iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl4UsEEACgkQHPApP6U8
pFjprxAAi60mVwH+LHo2HSCl+hwIhIyG2B9Dg2LjIJ8JPMA/WxaiDOOJCS+yMTbV
rVfdPrlT0B6Zd8ceTjz4ooZa79SwPFfCiFM97q1H/JwwsqVxaBEFEx6PgvnJzUUF
ZuJJEtQHijQgZo0gXv2plkqHTBrG5NMPNqQYEJ8aZqdjtSvtNkP2E03agVC/8SqW
mNyERNFcyOP3hUlNHSghPXl81ckSabqa83rLrwFCZQGJ2U71EnnietYxXT5Dz6Kx
W03z8HY2mClTETmZB/WvkCmG0F1AXQ8Xr2E4fJ2+meyNHgTZ2XjfYsKtZNKTQmiC
zlDgweuXuQ1r6DorLB4MUCm7HMffeDTwKEHBaYkIt7reHN8yGfT8sq1F8A0ZDKHi
y9Ugt0KwePPOGFK8mfST7lBWojPJL1wbyBVAYh+FL5f1hMScOdHRxbU9uz2p9NSB
RMubUWNCD1p8+sI8bLjQ//vU/iCLcWg7RStr/FSfXZEqjJv6EZ4OaNafahTcxvey
37Qz/eVTJQGeYa0+1rBvttVZJB6xrJwcscC3dgskTJ8VXJuAnwK0WdmMRzD7XLos
HP13SOoLXUgek07XH61OPq5dnbpUwq996GqpSLldLUJlCnbMi1vxkAmGe006zVXH
GWPoV1d4r7p0JjkyBlGQYUwiltuDFyNOx9uRS5FTaapaarhY6G0=
=Z571
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Ignore duplicate HTTP headers in Tomcat 8.5.50-0+deb9u1
Posted by Mark Thomas <ma...@apache.org>.
On 07/01/2020 07:10, Dennis Rech wrote:
> POST /foo HTTP/1.1
> Host: foo.com
> POST /foo HTTP/1.1
> Host: foo.com
> Content-[stuff] [...]
First two lines are OK.
The third line is going to be treated as an HTTP header. It is invalid
and Tomcat will reject it with a 400 response but you can tell Tomcat to
just ignore the invalid header with rejectIllegalHeaderName="false" on
the Connector.
The problem is going to be the second Host header.
RFC 7230 states:
<quote>
A server MUST respond with a 400 (Bad Request) status code to any
HTTP/1.1 request message that lacks a Host header field and to any
request message that contains more than one Host header field or a
Host header field with an invalid field-value.
</quote>
Any spec compliant server is almost certainly going to reject that
request. I guess a server might provide a hook for request modification
prior to rejection to allow the "fixing" of known invalid requests but
I'm not aware of any that do - at least not without going down the
writing a custom module route.
If we made Http11Processor.prepareRequest() protected then it would be
fairly simple to write a custom Processor that:
- extended Http11Processor
- overrode prepareRequest() to
a) remove the duplicate Host header
b) call super.prepareRequest()
I could provide one for you if you weren't comfortable doing that
yourself). However, even if we made the change now (which I'm happy to
do if you think it would be useful) it will take a while to filter
through to the Debian distribution.
There are several variations on this theme. One could write a custom
Processor for 8.5.50 that did the same thing - it would just be rather
more involved as one would have to copy rather more code from
Http11Processor.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Ignore duplicate HTTP headers in Tomcat 8.5.50-0+deb9u1
Posted by Dennis Rech <dr...@wikon.de>.
Hi Christopher,
Am 06.01.20 um 17:39 schrieb Christopher Schultz:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> Dennia,
>
> On 1/6/20 07:09, Dennis Rech wrote:
>> we have an application where HTTP clients have a kind of unclean
>> way of submitting HTTP POST requests to our tomcat server for data
>> upload: The |POST| and |Host: xxx| part appears twice in the
>> request.
> Yuck. You mean like this?
>
> POST /foo HTTP/1.1
> POST /foo HTTP/1.1
> Host: foo.com
> Host: foo.com
> Content-Type: application/x-www-url-encoded
> Content-Length: 13
>
> q=Hello World
>
> ?
No, rather like that:
POST /foo HTTP/1.1
Host: foo.com
POST /foo HTTP/1.1
Host: foo.com
Content-[stuff] [...]
>
>> Until now this didn't cause any problems with tomcat, but since
>> the latest release, Tomcat refuses to accept this message and
>> returns a 400 bad request immediately.
> Having two "host" headers should be okay. But repeating the request
> line is a clear violation of the HTTP spec that will be difficult to
> get over. I can't believe Tomcat ever allowed that, though it may have
> done so.
I read in the changelog that since Tomcat 8.5.22 it will also reply with
Bad request 400 if there are two Host fields in the header. But I guess
the double "POST" is even worse.
>
>> Unfortunately we'll not be able to change the client-side code. Is
>> there any way to tell the tomcat connector "ignore duplicate
>> headers" or so to make it work again? I guess the rewrite filters
>> for tomcat won't help as tomcat probably discards the incoming
>> message before handing it over to rewrite.
> Tomcat is responsible for reading the request line and routing the
> request to an application. If the request is broken badly enough, it
> won't be able to route.
>
> Headers are parsed as a part of that, and:
>
> POST /foo HTTP/1.1
>
> is not a valid header for at least two reasons:
>
> 1. There is no : character (required, even when the header has no value)
> 2. There are spaces in the "name" (the name is everything before colon )
Well, "POST"... is the actual request followed by the HTTP headers. POST
is not part of the actual header. Maybe I haven't pointed that out.
>
>> Example request:
>>
>> |POST /data/upload/test HTTP/1.1 Host: www.myhost.de:8180 POST
>> /data/upload/test HTTP/1.1 Host: www.myhost.de:8180 [...rest of
>> the request is ok...] |
> This got word-wrapped. Was this?
Yes I copied it from a formatted document, the pipes probably indicate
that this text was preformatted in the original document, sorry. Also
the newlines are missing.
POST /data/upload/test HTTP/1.1
Host: www.myhost.de:8180
POST/data/upload/test HTTP/1.1
Host: www.myhost.de:8180
[...here comes the remaining header with Content-Length etc followed by the body...]
>
>> POST /data/upload/test HTTP/1.1 Host: www.myhost.de:8180 POST
>> /data/upload/test HTTP/1.1 Host: www.myhost.de:8180 [...rest of the
>> request is ok...]
> ?
>
> Yikes. What kind of client is this?
It's a remote unit transmitting data to a server using POST file uploads
with - obviously - a little bug in the firmware that builds the HTTP
request manually as there is no curl library for the unit etc. which can
be used to generate the requests.
>
>> I wonder if there is a parameter for the |Connector| part in
>> server.xml or so to workaround this problem and restore the old
>> behaviour without downgrading.
> The good news is that the second POST could theoretically be
> considered to be a "broken" header and ignored. But Tomcat has been
> getting progressively more strict about what it will accept. There are
> all kinds of nasty ways to use malformed messages like this to confuse
> environments where e.g. a reverse-proxy and the origin server behave
> differently when they see requests like those above. It's better to
> just fail and fix the software. Why can't you fix the clients? Is this
> another case of internet-of-things garbage that can't practically be
> repaired?
Something like that. The devices could be updated in theory but probably
not over-the-air and many of them are already deployed somewhere in the
"wild" so we don't have physical access to them anymore. Unfortunately
we did not notice that in the past as tomcat always accepted these
requests until the latest update for debian came out.
I totally agree that the best solution would be to let those devices
send proper HTTP protocol but I guess we'll have to find a workaround on
server-side.
>
> - -chris
> -----BEGIN PGP SIGNATURE-----
> Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/
>
> iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl4TYrsACgkQHPApP6U8
> pFjNyRAAnebbKRBT99Zgk8vfnW4gUOGXYlgX8Xk2W8bUI7UPfqpphS6+t6Lwjti1
> SBnS2qKuUXELMI7fca5Iq2NhKD9cCxZhuWrRQ6+jSZESaT6EEGeKYFaJ3Jtn5Dv6
> ll6JKASuNTZtaDo+df0xYnDUvk6kT99aM3GY4FwYNLl/FIxv58D6YbUXN0TYLJQy
> 0IrnalQEQO8b8SiZcOPPAeA8ODZJQ37egAe3kP3xyWQPMl2u/966c0b711qZJ8JY
> D9JaR0fnVTZMHCHQmK9xLQKqra4T7uOv0UslQVheq47+SuXJea+qz1AmSJf3CU57
> bL4zD7XVKkcC/bsOu1uWUSaKjN0KRnIqwxwo2trpPyoMpKJBGY26GdekpM/jeLdU
> eyh4I8f8fJkJ7GerLZrZSGxKIlTs86fzM96wr/P4pgSclBjgCO3RMf9phl7URmgp
> rIqT/xjwjIyLWqASWhXUyT7F5EHbZLSvPRbvklAib2pwQU2taaeT4cZaWGB4wfWw
> +CXQgGj7E4bWmTeq4LsG3skJEVcYcVN/AHSsgWQ5MFsjKx5pxJjq++P+ZooTqQOr
> GNexi9KVFilovrf1iStoP4OgdPcf2o1l0PXCpbKf5i2woJgRpFS/oSlOno0KQaol
> hvddVhzgL8PYrcWywdGEIbzI/9eQm54nqOAMRlpOCTOaDPzbeLc=
> =vIL0
> -----END PGP SIGNATURE-----
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Ignore duplicate HTTP headers in Tomcat 8.5.50-0+deb9u1
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Dennia,
On 1/6/20 07:09, Dennis Rech wrote:
> we have an application where HTTP clients have a kind of unclean
> way of submitting HTTP POST requests to our tomcat server for data
> upload: The |POST| and |Host: xxx| part appears twice in the
> request.
Yuck. You mean like this?
POST /foo HTTP/1.1
POST /foo HTTP/1.1
Host: foo.com
Host: foo.com
Content-Type: application/x-www-url-encoded
Content-Length: 13
q=Hello World
?
> Until now this didn't cause any problems with tomcat, but since
> the latest release, Tomcat refuses to accept this message and
> returns a 400 bad request immediately.
Having two "host" headers should be okay. But repeating the request
line is a clear violation of the HTTP spec that will be difficult to
get over. I can't believe Tomcat ever allowed that, though it may have
done so.
> Unfortunately we'll not be able to change the client-side code. Is
> there any way to tell the tomcat connector "ignore duplicate
> headers" or so to make it work again? I guess the rewrite filters
> for tomcat won't help as tomcat probably discards the incoming
> message before handing it over to rewrite.
Tomcat is responsible for reading the request line and routing the
request to an application. If the request is broken badly enough, it
won't be able to route.
Headers are parsed as a part of that, and:
POST /foo HTTP/1.1
is not a valid header for at least two reasons:
1. There is no : character (required, even when the header has no value)
2. There are spaces in the "name" (the name is everything before colon )
> Example request:
>
> |POST /data/upload/test HTTP/1.1 Host: www.myhost.de:8180 POST
> /data/upload/test HTTP/1.1 Host: www.myhost.de:8180 [...rest of
> the request is ok...] |
This got word-wrapped. Was this?
> POST /data/upload/test HTTP/1.1 Host: www.myhost.de:8180 POST
> /data/upload/test HTTP/1.1 Host: www.myhost.de:8180 [...rest of the
> request is ok...]
?
Yikes. What kind of client is this?
> I wonder if there is a parameter for the |Connector| part in
> server.xml or so to workaround this problem and restore the old
> behaviour without downgrading.
The good news is that the second POST could theoretically be
considered to be a "broken" header and ignored. But Tomcat has been
getting progressively more strict about what it will accept. There are
all kinds of nasty ways to use malformed messages like this to confuse
environments where e.g. a reverse-proxy and the origin server behave
differently when they see requests like those above. It's better to
just fail and fix the software. Why can't you fix the clients? Is this
another case of internet-of-things garbage that can't practically be
repaired?
- -chris
-----BEGIN PGP SIGNATURE-----
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/
iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl4TYrsACgkQHPApP6U8
pFjNyRAAnebbKRBT99Zgk8vfnW4gUOGXYlgX8Xk2W8bUI7UPfqpphS6+t6Lwjti1
SBnS2qKuUXELMI7fca5Iq2NhKD9cCxZhuWrRQ6+jSZESaT6EEGeKYFaJ3Jtn5Dv6
ll6JKASuNTZtaDo+df0xYnDUvk6kT99aM3GY4FwYNLl/FIxv58D6YbUXN0TYLJQy
0IrnalQEQO8b8SiZcOPPAeA8ODZJQ37egAe3kP3xyWQPMl2u/966c0b711qZJ8JY
D9JaR0fnVTZMHCHQmK9xLQKqra4T7uOv0UslQVheq47+SuXJea+qz1AmSJf3CU57
bL4zD7XVKkcC/bsOu1uWUSaKjN0KRnIqwxwo2trpPyoMpKJBGY26GdekpM/jeLdU
eyh4I8f8fJkJ7GerLZrZSGxKIlTs86fzM96wr/P4pgSclBjgCO3RMf9phl7URmgp
rIqT/xjwjIyLWqASWhXUyT7F5EHbZLSvPRbvklAib2pwQU2taaeT4cZaWGB4wfWw
+CXQgGj7E4bWmTeq4LsG3skJEVcYcVN/AHSsgWQ5MFsjKx5pxJjq++P+ZooTqQOr
GNexi9KVFilovrf1iStoP4OgdPcf2o1l0PXCpbKf5i2woJgRpFS/oSlOno0KQaol
hvddVhzgL8PYrcWywdGEIbzI/9eQm54nqOAMRlpOCTOaDPzbeLc=
=vIL0
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org