Re: mod_ssl and Transfer-Encoding: chunked wastes ~58 bytes per chunk.

["Roy T. Fielding" <fi...@gbiv.com>]

On Aug 20, 2009, at 2:01 AM, Alex Stapleton wrote:
> 2009/8/19 Roy T. Fielding <fi...@gbiv.com>:
>> On Aug 19, 2009, at 9:53 AM, Alex Stapleton wrote:
>>
>>> (This has been cross posted to users@. I apologise if this mail  
>>> isn't
>>> relevant to the dev list.)
>>>
>>> First some background. We use Apache HTTPD 2.0 over a high-latency,
>>> high packet loss GPRS WAN. The cost per byte is tangible. We use  
>>> SSL.
>>> We also use Transfer-Encoding: chunked sometimes. This is a machine
>>> monitoring application. We are using iframe streaming to push real
>>> time data to operators browsers.
>>>
>>> I have noticed after much faffing around with wireshark that httpd
>>> will transmit 3 Application Data fragments for each chunk in a  
>>> chunked
>>> HTTP stream. The fragments are the HTTP chunk size, the chunk data,
>>> and then a CRLF. All fragments in an SSL session are rounded to the
>>> length of the ciphers block size. This results in the 4+2 bytes for
>>> the chunk frame ending up being 64 bytes of data over TCP. A  
>>> waste of
>>> 58 bytes per chunk in this case.
>>
>> That's odd -- we don't do this with non-SSL writes (last I checked).
>> Perhaps we are relying on a TCP cork for the non-SSL case?
>> What is your operating system and platform?
>
> I initially discovered this issue on a fairly old Ubuntu 6 machine
> running httpd 2.0.55-4ubuntu4.1. I have since recreated it on my OS X
> 10.5 iMac using httpd 2.0.64 from MacPorts. I suppose I should also
> point out that I am using PHP 5.2.9 to generate the chunked data. If
> there's a way of doing chunked output using "plain" Apache let me know
> and I can test that too if needed.

Er, have you tested it with 2.2.x?  The likelihood of us making any
non-security changes to the 2.0.x branch is extremely small.

Apache does chunked output by default if no content-length is
provided and the client says it is HTTP/1.1.

....Roy

Re: mod_ssl and Transfer-Encoding: chunked wastes ~58 bytes per chunk.

[Alex Stapleton <al...@gmail.com>]

2009/8/21 Alex Stapleton <al...@gmail.com>:
> 2009/8/20 Roy T. Fielding <fi...@gbiv.com>:
>> On Aug 20, 2009, at 2:01 AM, Alex Stapleton wrote:
>>>
>>> 2009/8/19 Roy T. Fielding <fi...@gbiv.com>:
>>>>
>>>> On Aug 19, 2009, at 9:53 AM, Alex Stapleton wrote:
>>>>
>>>>> (This has been cross posted to users@. I apologise if this mail isn't
>>>>> relevant to the dev list.)
>>>>>
>>>>> First some background. We use Apache HTTPD 2.0 over a high-latency,
>>>>> high packet loss GPRS WAN. The cost per byte is tangible. We use SSL.
>>>>> We also use Transfer-Encoding: chunked sometimes. This is a machine
>>>>> monitoring application. We are using iframe streaming to push real
>>>>> time data to operators browsers.
>>>>>
>>>>> I have noticed after much faffing around with wireshark that httpd
>>>>> will transmit 3 Application Data fragments for each chunk in a chunked
>>>>> HTTP stream. The fragments are the HTTP chunk size, the chunk data,
>>>>> and then a CRLF. All fragments in an SSL session are rounded to the
>>>>> length of the ciphers block size. This results in the 4+2 bytes for
>>>>> the chunk frame ending up being 64 bytes of data over TCP. A waste of
>>>>> 58 bytes per chunk in this case.
>>>>
>>>> That's odd -- we don't do this with non-SSL writes (last I checked).
>>>> Perhaps we are relying on a TCP cork for the non-SSL case?
>>>> What is your operating system and platform?
>>>
>>> I initially discovered this issue on a fairly old Ubuntu 6 machine
>>> running httpd 2.0.55-4ubuntu4.1. I have since recreated it on my OS X
>>> 10.5 iMac using httpd 2.0.64 from MacPorts. I suppose I should also
>>> point out that I am using PHP 5.2.9 to generate the chunked data. If
>>> there's a way of doing chunked output using "plain" Apache let me know
>>> and I can test that too if needed.
>>
>> Er, have you tested it with 2.2.x?  The likelihood of us making any
>> non-security changes to the 2.0.x branch is extremely small.
>
> I have tested with 2.2.11 from MacPorts on my iMac and it also
> exhibits this behaviour. I'll try and do a 2.2.13 build to test with
> :)

Confirmed on 2.2.13.

>> Apache does chunked output by default if no content-length is
>> provided and the client says it is HTTP/1.1.
>>
>> ....Roy
>>
>>
>

Re: mod_ssl and Transfer-Encoding: chunked wastes ~58 bytes per chunk.

[Alex Stapleton <al...@gmail.com>]

2009/8/20 Roy T. Fielding <fi...@gbiv.com>:
> On Aug 20, 2009, at 2:01 AM, Alex Stapleton wrote:
>>
>> 2009/8/19 Roy T. Fielding <fi...@gbiv.com>:
>>>
>>> On Aug 19, 2009, at 9:53 AM, Alex Stapleton wrote:
>>>
>>>> (This has been cross posted to users@. I apologise if this mail isn't
>>>> relevant to the dev list.)
>>>>
>>>> First some background. We use Apache HTTPD 2.0 over a high-latency,
>>>> high packet loss GPRS WAN. The cost per byte is tangible. We use SSL.
>>>> We also use Transfer-Encoding: chunked sometimes. This is a machine
>>>> monitoring application. We are using iframe streaming to push real
>>>> time data to operators browsers.
>>>>
>>>> I have noticed after much faffing around with wireshark that httpd
>>>> will transmit 3 Application Data fragments for each chunk in a chunked
>>>> HTTP stream. The fragments are the HTTP chunk size, the chunk data,
>>>> and then a CRLF. All fragments in an SSL session are rounded to the
>>>> length of the ciphers block size. This results in the 4+2 bytes for
>>>> the chunk frame ending up being 64 bytes of data over TCP. A waste of
>>>> 58 bytes per chunk in this case.
>>>
>>> That's odd -- we don't do this with non-SSL writes (last I checked).
>>> Perhaps we are relying on a TCP cork for the non-SSL case?
>>> What is your operating system and platform?
>>
>> I initially discovered this issue on a fairly old Ubuntu 6 machine
>> running httpd 2.0.55-4ubuntu4.1. I have since recreated it on my OS X
>> 10.5 iMac using httpd 2.0.64 from MacPorts. I suppose I should also
>> point out that I am using PHP 5.2.9 to generate the chunked data. If
>> there's a way of doing chunked output using "plain" Apache let me know
>> and I can test that too if needed.
>
> Er, have you tested it with 2.2.x?  The likelihood of us making any
> non-security changes to the 2.0.x branch is extremely small.

I have tested with 2.2.11 from MacPorts on my iMac and it also
exhibits this behaviour. I'll try and do a 2.2.13 build to test with
:)

> Apache does chunked output by default if no content-length is
> provided and the client says it is HTTP/1.1.
>
> ....Roy
>
>

Re: mod_ssl and Transfer-Encoding: chunked wastes ~58 bytes per chunk.

["William A. Rowe, Jr." <wr...@rowe-clan.net>]

Roy T. Fielding wrote:
> 
> Apache does chunked output by default if no content-length is
> provided and the client says it is HTTP/1.1.

And further processes chunked input; it's not up to PHP or other
user agents to process this further, it's on the webserver to do so.

If you want to [de]chunk your own data, use mod_proxy_http.