You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by David Cleary <da...@progress.com> on 2026/02/24 13:43:45 UTC
Access log Bytes Written when compression is enabled
A customer is asking why the number of bytes written can appear as - in the access log when compression is enabled. They use this value in one of their monitoring tools.
AI tells me this:
When compression is enabled:
1. Compressed data is buffered in the compression filter layer
2. getBytesWritten(false) returns the count of bytes that have already been written to the socket
3. Any compressed data still residing in the buffer is not included because the buffer was never flushed
4. This can result in zero or an inaccurate byte count being logged
Is there any way to ensure this value gets logged when compression is enabled?
Thanks
Dave
Re: Access log Bytes Written when compression is enabled
Posted by Mark Thomas <ma...@apache.org>.
On 02/03/2026 21:01, David Cleary wrote:
>
>
> On 2/24/26 8:43 AM, David Cleary wrote:
>> A customer is asking why the number of bytes written can appear as - in the access log when compression is enabled. They use this value in one of their monitoring tools.
>>
>> AI tells me this:
>>
>> When compression is enabled:
>>
>> 1. Compressed data is buffered in the compression filter layer
>> 2. getBytesWritten(false) returns the count of bytes that have already been written to the socket
>> 3. Any compressed data still residing in the buffer is not included because the buffer was never flushed
>> 4. This can result in zero or an inaccurate byte count being logged
The bit AI missed is that the response is complete by the time the
access log valve is called so the bytes should be correct.
>> Is there any way to ensure this value gets logged when compression is enabled?
>
>> It sounds like your report says that only *sometimes* is the value
>> appearing as - in the logs. Can you confirm that? Can you tell us
>> roughly what percentage of access log entries contain - versus a real value?
An answer to the above question would be helpful.
>> Is there any pattern to the URLs that are being requested that show - as
>> the bytes-written value?
And this one.
>> My initial reaction is that Tomcat should be flushing that buffer even
>> if the application isn't doing it, and should therefore be able to give
>> an accurate number.
Agreed.
>> What version are you using, and are there any unusual things going on
>> such as other Filters that might be altering the number of bytes passing
>> through the pipeline?
Also,
- Connector configuration
- MIME type of file
- size of file before compression
I've tried and failed to reproduce this. Without a minimal reproducible
test case, there is very little we can do.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Access log Bytes Written when compression is enabled
Posted by Christopher Schultz <ch...@christopherschultz.net>.
David,
On 3/2/26 4:01 PM, David Cleary wrote:
>
>
> On 2/24/26 8:43 AM, David Cleary wrote:
>> A customer is asking why the number of bytes written can appear as - in the access log when compression is enabled. They use this value in one of their monitoring tools.
>>
>> AI tells me this:
>>
>> When compression is enabled:
>>
>> 1. Compressed data is buffered in the compression filter layer
>> 2. getBytesWritten(false) returns the count of bytes that have already been written to the socket
>> 3. Any compressed data still residing in the buffer is not included because the buffer was never flushed
>> 4. This can result in zero or an inaccurate byte count being logged
>>
>> Is there any way to ensure this value gets logged when compression is enabled?
>
>> It sounds like your report says that only *sometimes* is the value
>> appearing as - in the logs. Can you confirm that? Can you tell us
>> roughly what percentage of access log entries contain - versus a real value?
>
>> Is there any pattern to the URLs that are being requested that show - as
>> the bytes-written value?
>
>> My initial reaction is that Tomcat should be flushing that buffer even
>> if the application isn't doing it, and should therefore be able to give
>> an accurate number.
>
>> What version are you using, and are there any unusual things going on
>> such as other Filters that might be altering the number of bytes passing
>> through the pipeline?
>
>> -chris
>
> Sorry. My tech support guys went dark with some clarifying questions. This is Tomcat 10.1.x. Not sure of the exact version they reproduced on because we patch Tomcat regularly. Here are two concurrent requests, one with a browser and one with curl that show the issue.
>
> From the browser (payload is - i.e. not expected ): 127.0.0.1 - anonymousUser "GET /web/tsCustomer.p HTTP/1.1" 200 - ROOT:w:0000003e 82
>
> With curl ( payload is 500540 i.e. expected ): 0:0:0:0:0:0:0:1 - anonymousUser "GET /web/tsCustomer.p HTTP/1.1" 200 500540 ROOT:w:0000003f 83
>
> I would be shocked if IPv4 vs IPv6 has anything to do with it. I do not know if the curl request negotiated compression, but am pretty sure the browser did. It isn't the bytes being off. It is them being missing. And sorry for any formatting issues. Not top posting with Outlook is hard. If I get anything more, I will update.
I'm afraid I'm still confused. Your browser request contained no request
body and so you expected to see "-" in the log file, and you got it. So
that request is good, right?
Then the next request (from curl), you have 500540 bytes of request
entity and the log file says "500540". So that's good, too, right?
Why does a GET request return no response? Payload is "-" -- what does
that mean?
I think I'm more confused now than I was at first.
-chris
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Access log Bytes Written when compression is enabled
Posted by David Cleary <da...@progress.com>.
On 2/24/26 8:43 AM, David Cleary wrote:
> A customer is asking why the number of bytes written can appear as - in the access log when compression is enabled. They use this value in one of their monitoring tools.
>
> AI tells me this:
>
> When compression is enabled:
>
> 1. Compressed data is buffered in the compression filter layer
> 2. getBytesWritten(false) returns the count of bytes that have already been written to the socket
> 3. Any compressed data still residing in the buffer is not included because the buffer was never flushed
> 4. This can result in zero or an inaccurate byte count being logged
>
> Is there any way to ensure this value gets logged when compression is enabled?
>It sounds like your report says that only *sometimes* is the value
>appearing as - in the logs. Can you confirm that? Can you tell us
>roughly what percentage of access log entries contain - versus a real value?
>Is there any pattern to the URLs that are being requested that show - as
>the bytes-written value?
>My initial reaction is that Tomcat should be flushing that buffer even
>if the application isn't doing it, and should therefore be able to give
>an accurate number.
>What version are you using, and are there any unusual things going on
>such as other Filters that might be altering the number of bytes passing
>through the pipeline?
>-chris
Sorry. My tech support guys went dark with some clarifying questions. This is Tomcat 10.1.x. Not sure of the exact version they reproduced on because we patch Tomcat regularly. Here are two concurrent requests, one with a browser and one with curl that show the issue.
From the browser (payload is - i.e. not expected ): 127.0.0.1 - anonymousUser "GET /web/tsCustomer.p HTTP/1.1" 200 - ROOT:w:0000003e 82
With curl ( payload is 500540 i.e. expected ): 0:0:0:0:0:0:0:1 - anonymousUser "GET /web/tsCustomer.p HTTP/1.1" 200 500540 ROOT:w:0000003f 83
I would be shocked if IPv4 vs IPv6 has anything to do with it. I do not know if the curl request negotiated compression, but am pretty sure the browser did. It isn't the bytes being off. It is them being missing. And sorry for any formatting issues. Not top posting with Outlook is hard. If I get anything more, I will update.
Thanks
Dave
Re: Access log Bytes Written when compression is enabled
Posted by Christopher Schultz <ch...@christopherschultz.net>.
Dave,
On 2/24/26 8:43 AM, David Cleary wrote:
> A customer is asking why the number of bytes written can appear as - in the access log when compression is enabled. They use this value in one of their monitoring tools.
>
> AI tells me this:
>
> When compression is enabled:
>
> 1. Compressed data is buffered in the compression filter layer
> 2. getBytesWritten(false) returns the count of bytes that have already been written to the socket
> 3. Any compressed data still residing in the buffer is not included because the buffer was never flushed
> 4. This can result in zero or an inaccurate byte count being logged
>
> Is there any way to ensure this value gets logged when compression is enabled?
It sounds like your report says that only *sometimes* is the value
appearing as - in the logs. Can you confirm that? Can you tell us
roughly what percentage of access log entries contain - versus a real value?
Is there any pattern to the URLs that are being requested that show - as
the bytes-written value?
My initial reaction is that Tomcat should be flushing that buffer even
if the application isn't doing it, and should therefore be able to give
an accurate number.
What version are you using, and are there any unusual things going on
such as other Filters that might be altering the number of bytes passing
through the pipeline?
-chris
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org