Re: [OT] Again with the missing headers

[Christopher Schultz <ch...@christopherschultz.net>]

Mark,

On 4/12/21 04:16, Mark Thomas wrote:
> maxHttpHeaderSize only applies to Tomcat reading requests from clients. 
> It has no impact on the headers Tomcat sends to the client.
> 
> Given that the issue is size dependent and the the header is missing 
> only on the larger responses, I would guess that the Servlet is writing 
> the header after the response has been committed in those cases and is, 
> therefore, being lost.
> 
> In HTTP, headers usually come before the body. When generating a 
> response, Tomcat will buffer the response body until either there is an 
> explicit flush or explicit commit or the buffer is full at which point 
> an implicit flush occurs.
> 
> When a flush (implicit or explicit) occurs, if the response is not 
> committed at that point then the response is automatically committed. 
> Committing the response == writing the response headers.
> 
> You typically want to ensure that the Servlet writes all the headers 
> before it writes the response body.
> 
> There are alternative options:
> 1. Use a large enough response buffer to buffer the entire response. 
> Keep in mind you need to have enough free memory for every concurrent 
> request to have a buffer this large whether it uses it or not.
> 
> 2. Servlet 4 (Tomcat 9 onwards) added support for trailer headers (see 
> RFC 7230 section 4.4). These can be used with chunked encoding and 
> allows header values to be written after the response body. You might 
> want to confirm that the client you are using supports trailer headers.

Notably the HttpClient client class provided by Java 9+ *does* support 
trailers in the response from the server. Unfortunately, it doesn't 
support sending trailers *to* the server, which is a shame.

-chris

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: [OT] Again with the missing headers

[Rob Sargent <rs...@xmission.com>]

> On Apr 12, 2021, at 2:04 PM, Christopher Schultz <ch...@christopherschultz.net> wrote:
> 
> Mark,
> 
> On 4/12/21 04:16, Mark Thomas wrote:
>> maxHttpHeaderSize only applies to Tomcat reading requests from clients. It has no impact on the headers Tomcat sends to the client.
>> Given that the issue is size dependent and the the header is missing only on the larger responses, I would guess that the Servlet is writing the header after the response has been committed in those cases and is, therefore, being lost.
>> In HTTP, headers usually come before the body. When generating a response, Tomcat will buffer the response body until either there is an explicit flush or explicit commit or the buffer is full at which point an implicit flush occurs.
>> When a flush (implicit or explicit) occurs, if the response is not committed at that point then the response is automatically committed. Committing the response == writing the response headers.
>> You typically want to ensure that the Servlet writes all the headers before it writes the response body.
>> There are alternative options:
>> 1. Use a large enough response buffer to buffer the entire response. Keep in mind you need to have enough free memory for every concurrent request to have a buffer this large whether it uses it or not.
>> 2. Servlet 4 (Tomcat 9 onwards) added support for trailer headers (see RFC 7230 section 4.4). These can be used with chunked encoding and allows header values to be written after the response body. You might want to confirm that the client you are using supports trailer headers.
> 
> Notably the HttpClient client class provided by Java 9+ *does* support trailers in the response from the server. Unfortunately, it doesn't support sending trailers *to* the server, which is a shame.
> 
> -chris

I can live with to-client only!  Now to figure out the setting.
Thanks all.

> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org <ma...@tomcat.apache.org>
> For additional commands, e-mail: users-help@tomcat.apache.org <ma...@tomcat.apache.org>