You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Amit Pande <Am...@veritas.com> on 2018/11/02 21:16:45 UTC
Request header too large..
As per current implementation (below snippet is from 8.5.28), if the request header is too large (by default >8K, the default maxHttpHeaderSize), is thrown below error message is seen.
For request --
02-Nov-2018 15:15:47.649 INFO [catalina-exec-40] org.apache.coyote.http11.Http11Processor.service Error parsing HTTP request header
Note: further occurrences of HTTP header parsing errors will be logged at DEBUG level.
java.lang.IllegalArgumentException: Request header is too large
at org.apache.coyote.http11.Http11InputBuffer.parseHeaders(Http11InputBuffer.java:575)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:703)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:790)
For response—
02-Nov-2018 15:15:47.867 SEVERE [catalina-exec-40] org.apache.coyote.http11.Http11Processor.endRequest Error finishing response
org.apache.coyote.http11.HeadersTooLargeException: An attempt was made to write more data to the response headers than there was room available in the buffer. Increase maxHttpHeaderSize on the connector or write less data into the response headers.
at org.apache.coyote.http11.Http11OutputBuffer.checkLengthBeforeWrite(Http11OutputBuffer.java:544)
at org.apache.coyote.http11.Http11OutputBuffer.write(Http11OutputBuffer.java:481)
at org.apache.coyote.http11.Http11OutputBuffer.sendStatus(Http11OutputBuffer.java:361)
at org.apache.coyote.http11.Http11Processor.prepareResponse(Http11Processor.java:1339)
at org.apache.coyote.AbstractProcessor.action(AbstractProcessor.java:260)
at org.apache.coyote.http11.Http11Processor.endRequest(Http11Processor.java:1523)
However, this doesn’t tell us the name of the offending header which I think is valuable in debugging the issues.
Do you feel it’s worth the enhancement request?
Also, I noticed the comments
“
// (1) Headers plus request line size does not exceed its limit
// (2) There are enough bytes to avoid expanding the buffer when
// reading body
“
It says request line size ..does that mean maxHttpHeaderSize comes in play for request line (which includes query params) as well?
Thanks,
Amit
Re: [EXTERNAL] Re: Request header too large..
Posted by Mark Thomas <ma...@apache.org>.
On 06/11/2018 00:02, Amit Pande wrote:
> Thanks Mark for the update. Seems like there is a potential bug in our client components which happened to keep on adding to existing header list.
>
> https://tomcat.apache.org/tomcat-8.5-doc/config/http.html
>
> From the documentation:
>
> maxHttpHeaderSize :The maximum size of the request and response HTTP header, specified in bytes. If not specified, this attribute is set to 8192 (8 KB).
>
> Although some online searches (e.g. https://serverfault.com/questions/56691/whats-the-maximum-url-length-in-tomcat) suggest that this maxHttpHeaderSize applies to request line as well, could the documentation be made more elaborate?
>
> Can we mention that this attribute puts the upper bound on combined size of all request headers and the request line?
Sure. Patch / pull request welcome.
Mark
>
> Thanks,
> Amit
>
> On 11/3/18, 4:39 AM, "Mark Thomas" <ma...@apache.org> wrote:
>
> On 03/11/2018 01:35, Amit Pande wrote:
> > Thanks Chris. Yes, I will soon send out the patch for review.
>
> Logically, I can see some hurdles for such a patch that are going to be
> difficult to overcome.
>
> Tomcat will process all of the received data up to the limit. It is on
> the subsequent read to get more data that the exception is thrown.
>
> The issue is how to determine which header is actually the problem as it
> may not be the one currently being read.
>
> Consider the following:
>
> ReqLine - 1024 bytes
> Headers - 5 * 128 bytes
> Header - 1 * 6*1024 bytes
> Headers - 5 * 128 bytes
>
> Is the problem the 6k header? The 1k request line? Has the client
> duplicated the block of 5 128 byte headers? Or is the request valid and
> Tomcat's limit too low?
>
> There is no way for Tomcat to determine where the root cause lies. The
> only reliable way to figure out what went wrong is for a knowledgeable
> system admin to look at the complete set of headers.
>
> Mark
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: [EXTERNAL] Re: Request header too large..
Posted by Amit Pande <Am...@veritas.com>.
Thanks Mark for the update. Seems like there is a potential bug in our client components which happened to keep on adding to existing header list.
https://tomcat.apache.org/tomcat-8.5-doc/config/http.html
From the documentation:
maxHttpHeaderSize :The maximum size of the request and response HTTP header, specified in bytes. If not specified, this attribute is set to 8192 (8 KB).
Although some online searches (e.g. https://serverfault.com/questions/56691/whats-the-maximum-url-length-in-tomcat) suggest that this maxHttpHeaderSize applies to request line as well, could the documentation be made more elaborate?
Can we mention that this attribute puts the upper bound on combined size of all request headers and the request line?
Thanks,
Amit
On 11/3/18, 4:39 AM, "Mark Thomas" <ma...@apache.org> wrote:
On 03/11/2018 01:35, Amit Pande wrote:
> Thanks Chris. Yes, I will soon send out the patch for review.
Logically, I can see some hurdles for such a patch that are going to be
difficult to overcome.
Tomcat will process all of the received data up to the limit. It is on
the subsequent read to get more data that the exception is thrown.
The issue is how to determine which header is actually the problem as it
may not be the one currently being read.
Consider the following:
ReqLine - 1024 bytes
Headers - 5 * 128 bytes
Header - 1 * 6*1024 bytes
Headers - 5 * 128 bytes
Is the problem the 6k header? The 1k request line? Has the client
duplicated the block of 5 128 byte headers? Or is the request valid and
Tomcat's limit too low?
There is no way for Tomcat to determine where the root cause lies. The
only reliable way to figure out what went wrong is for a knowledgeable
system admin to look at the complete set of headers.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: [EXTERNAL] Re: Request header too large..
Posted by Mark Thomas <ma...@apache.org>.
On 03/11/2018 01:35, Amit Pande wrote:
> Thanks Chris. Yes, I will soon send out the patch for review.
Logically, I can see some hurdles for such a patch that are going to be
difficult to overcome.
Tomcat will process all of the received data up to the limit. It is on
the subsequent read to get more data that the exception is thrown.
The issue is how to determine which header is actually the problem as it
may not be the one currently being read.
Consider the following:
ReqLine - 1024 bytes
Headers - 5 * 128 bytes
Header - 1 * 6*1024 bytes
Headers - 5 * 128 bytes
Is the problem the 6k header? The 1k request line? Has the client
duplicated the block of 5 128 byte headers? Or is the request valid and
Tomcat's limit too low?
There is no way for Tomcat to determine where the root cause lies. The
only reliable way to figure out what went wrong is for a knowledgeable
system admin to look at the complete set of headers.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: [EXTERNAL] Re: Request header too large..
Posted by Amit Pande <Am...@veritas.com>.
Thanks Chris. Yes, I will soon send out the patch for review.
Thanks,
Amit
On 11/2/18, 5:25 PM, "Christopher Schultz" <ch...@christopherschultz.net> wrote:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Amit,
On 11/2/18 17:16, Amit Pande wrote:
> As per current implementation (below snippet is from 8.5.28), if
> the request header is too large (by default >8K, the default
> maxHttpHeaderSize), is thrown below error message is seen.
>
>
> For request -- 02-Nov-2018 15:15:47.649 INFO [catalina-exec-40]
> org.apache.coyote.http11.Http11Processor.service Error parsing HTTP
> request header Note: further occurrences of HTTP header parsing
> errors will be logged at DEBUG level.
> java.lang.IllegalArgumentException: Request header is too large at
> org.apache.coyote.http11.Http11InputBuffer.parseHeaders(Http11InputBuf
fer.java:575)
>
>
at
org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:70
3)
> at
> org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLigh
t.java:66)
>
>
at
org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractPro
tocol.java:790)
>
> For response—
>
> 02-Nov-2018 15:15:47.867 SEVERE [catalina-exec-40]
> org.apache.coyote.http11.Http11Processor.endRequest Error finishing
> response org.apache.coyote.http11.HeadersTooLargeException: An
> attempt was made to write more data to the response headers than
> there was room available in the buffer. Increase maxHttpHeaderSize
> on the connector or write less data into the response headers. at
> org.apache.coyote.http11.Http11OutputBuffer.checkLengthBeforeWrite(Htt
p11OutputBuffer.java:544)
>
>
at
org.apache.coyote.http11.Http11OutputBuffer.write(Http11OutputBuffer.jav
a:481)
> at
> org.apache.coyote.http11.Http11OutputBuffer.sendStatus(Http11OutputBuf
fer.java:361)
>
>
at
org.apache.coyote.http11.Http11Processor.prepareResponse(Http11Processor
.java:1339)
> at
> org.apache.coyote.AbstractProcessor.action(AbstractProcessor.java:260)
>
>
at
org.apache.coyote.http11.Http11Processor.endRequest(Http11Processor.java
:1523)
>
>
> However, this doesn’t tell us the name of the offending header
> which I think is valuable in debugging the issues.
>
> Do you feel it’s worth the enhancement request?
>
> Also, I noticed the comments
>
> “ // (1) Headers plus request line size does not exceed its limit
>
>
>
> // (2) There are enough bytes to avoid expanding the buffer when
>
>
> // reading body “
>
> It says request line size ..does that mean maxHttpHeaderSize comes
> in play for request line (which includes query params) as well?
Yes.
The request headers are read before selecting which context will
service the request. The request line, which not formally a part of
the "headers" per the HTTP spec, are considered to be "part of the
headers" as far as Tomcat is concerned, because they all share the
same buffer.
To locate the offending header, look at the end of the request, then
re-wind to the previous "\r\nheader:". If you don't find
"\r\nheader:", then it was the request line. But it could be request +
partial-header-name, so you'd have to be careful.
I'm not sure it's worth it to add a debugging enhancement (you
probably already know what your huge headers are), but I am by no
means the King of Tomcat: feel free to request that enhancement in BZ
and see if you can get some support.
If you write a patch/PR for it, it will almost certainly be added as
long as its not too intrusive :)
- -chris
-----BEGIN PGP SIGNATURE-----
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/
iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlvcztQACgkQHPApP6U8
pFhj+Q//QIMUy44I2SDH5kM7UeMk2SCB/5J5hW79RGfIGITFpmEMttBONF8wilg7
U7ldyjDuu2Msn+ynRPVg5lxf9pq3OQJKySv7xFc3AG4C44em4xRrJI8aHGsLxH5N
dQyyJY/oP61HzeiiiyDLYxQY3sidC+4gieOEJ2oG5ZsXJwFpvfc0Q1LyiIzN8fXa
o2PkH0B2kPsksgXD8WZLPXAFMgAUdmUu0wVzM0lt/kBvlG9LGW/vEd+fVeLrG93s
UEs3IrWT2aTWnOVyZl2LAPhL2GynfrDUmsmsc8EF6kljXiX3IWphaH9z1nbCpLFM
zDsUoIRpqm1sgXObV52zW8YbnaUzIEAZaI7lIlrOctRvc9ACeCvKkIc+aVylgnmR
zpEhFz/gOchl1T6x2+MXLHavch653F8OgYY52/TCqxQheVKSHjc2jJu9UTcb+7z0
EBbfeWvUeEdNd7edGORhKx6tJLotbRhF1Yr2sjxxkY8I31RGHv+ZlNStOuxr5uoI
0c68L2XGbe9gBaLbjopYOfb+Y8y73iX3wfJPKIlvXl8LkupA1wyg0NAwOn+ZDL2E
Rl865F3+DaaVaFoSRkpgp7pUrKfh9wi/N2CdCJ13hH6WY32/JD8+Z2X+YuRfMu6+
6dh+NiVvrQc998/gutkx+OnH+RTC0STJp6RTzYNZbnL5hi5z1Ms=
=bjZe
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Request header too large..
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Amit,
On 11/2/18 17:16, Amit Pande wrote:
> As per current implementation (below snippet is from 8.5.28), if
> the request header is too large (by default >8K, the default
> maxHttpHeaderSize), is thrown below error message is seen.
>
>
> For request -- 02-Nov-2018 15:15:47.649 INFO [catalina-exec-40]
> org.apache.coyote.http11.Http11Processor.service Error parsing HTTP
> request header Note: further occurrences of HTTP header parsing
> errors will be logged at DEBUG level.
> java.lang.IllegalArgumentException: Request header is too large at
> org.apache.coyote.http11.Http11InputBuffer.parseHeaders(Http11InputBuf
fer.java:575)
>
>
at
org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:70
3)
> at
> org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLigh
t.java:66)
>
>
at
org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractPro
tocol.java:790)
>
> For response—
>
> 02-Nov-2018 15:15:47.867 SEVERE [catalina-exec-40]
> org.apache.coyote.http11.Http11Processor.endRequest Error finishing
> response org.apache.coyote.http11.HeadersTooLargeException: An
> attempt was made to write more data to the response headers than
> there was room available in the buffer. Increase maxHttpHeaderSize
> on the connector or write less data into the response headers. at
> org.apache.coyote.http11.Http11OutputBuffer.checkLengthBeforeWrite(Htt
p11OutputBuffer.java:544)
>
>
at
org.apache.coyote.http11.Http11OutputBuffer.write(Http11OutputBuffer.jav
a:481)
> at
> org.apache.coyote.http11.Http11OutputBuffer.sendStatus(Http11OutputBuf
fer.java:361)
>
>
at
org.apache.coyote.http11.Http11Processor.prepareResponse(Http11Processor
.java:1339)
> at
> org.apache.coyote.AbstractProcessor.action(AbstractProcessor.java:260)
>
>
at
org.apache.coyote.http11.Http11Processor.endRequest(Http11Processor.java
:1523)
>
>
> However, this doesn’t tell us the name of the offending header
> which I think is valuable in debugging the issues.
>
> Do you feel it’s worth the enhancement request?
>
> Also, I noticed the comments
>
> “ // (1) Headers plus request line size does not exceed its limit
>
>
>
> // (2) There are enough bytes to avoid expanding the buffer when
>
>
> // reading body “
>
> It says request line size ..does that mean maxHttpHeaderSize comes
> in play for request line (which includes query params) as well?
Yes.
The request headers are read before selecting which context will
service the request. The request line, which not formally a part of
the "headers" per the HTTP spec, are considered to be "part of the
headers" as far as Tomcat is concerned, because they all share the
same buffer.
To locate the offending header, look at the end of the request, then
re-wind to the previous "\r\nheader:". If you don't find
"\r\nheader:", then it was the request line. But it could be request +
partial-header-name, so you'd have to be careful.
I'm not sure it's worth it to add a debugging enhancement (you
probably already know what your huge headers are), but I am by no
means the King of Tomcat: feel free to request that enhancement in BZ
and see if you can get some support.
If you write a patch/PR for it, it will almost certainly be added as
long as its not too intrusive :)
- -chris
-----BEGIN PGP SIGNATURE-----
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/
iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlvcztQACgkQHPApP6U8
pFhj+Q//QIMUy44I2SDH5kM7UeMk2SCB/5J5hW79RGfIGITFpmEMttBONF8wilg7
U7ldyjDuu2Msn+ynRPVg5lxf9pq3OQJKySv7xFc3AG4C44em4xRrJI8aHGsLxH5N
dQyyJY/oP61HzeiiiyDLYxQY3sidC+4gieOEJ2oG5ZsXJwFpvfc0Q1LyiIzN8fXa
o2PkH0B2kPsksgXD8WZLPXAFMgAUdmUu0wVzM0lt/kBvlG9LGW/vEd+fVeLrG93s
UEs3IrWT2aTWnOVyZl2LAPhL2GynfrDUmsmsc8EF6kljXiX3IWphaH9z1nbCpLFM
zDsUoIRpqm1sgXObV52zW8YbnaUzIEAZaI7lIlrOctRvc9ACeCvKkIc+aVylgnmR
zpEhFz/gOchl1T6x2+MXLHavch653F8OgYY52/TCqxQheVKSHjc2jJu9UTcb+7z0
EBbfeWvUeEdNd7edGORhKx6tJLotbRhF1Yr2sjxxkY8I31RGHv+ZlNStOuxr5uoI
0c68L2XGbe9gBaLbjopYOfb+Y8y73iX3wfJPKIlvXl8LkupA1wyg0NAwOn+ZDL2E
Rl865F3+DaaVaFoSRkpgp7pUrKfh9wi/N2CdCJ13hH6WY32/JD8+Z2X+YuRfMu6+
6dh+NiVvrQc998/gutkx+OnH+RTC0STJp6RTzYNZbnL5hi5z1Ms=
=bjZe
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org