Re: mod_ssl and Transfer-Encoding: chunked wastes ~58 bytes per chunk.

[Alex Stapleton <al...@gmail.com>]

2009/8/19 Roy T. Fielding <fi...@gbiv.com>:
> On Aug 19, 2009, at 9:53 AM, Alex Stapleton wrote:
>
>> (This has been cross posted to users@. I apologise if this mail isn't
>> relevant to the dev list.)
>>
>> First some background. We use Apache HTTPD 2.0 over a high-latency,
>> high packet loss GPRS WAN. The cost per byte is tangible. We use SSL.
>> We also use Transfer-Encoding: chunked sometimes. This is a machine
>> monitoring application. We are using iframe streaming to push real
>> time data to operators browsers.
>>
>> I have noticed after much faffing around with wireshark that httpd
>> will transmit 3 Application Data fragments for each chunk in a chunked
>> HTTP stream. The fragments are the HTTP chunk size, the chunk data,
>> and then a CRLF. All fragments in an SSL session are rounded to the
>> length of the ciphers block size. This results in the 4+2 bytes for
>> the chunk frame ending up being 64 bytes of data over TCP. A waste of
>> 58 bytes per chunk in this case.
>
> That's odd -- we don't do this with non-SSL writes (last I checked).
> Perhaps we are relying on a TCP cork for the non-SSL case?
> What is your operating system and platform?

I initially discovered this issue on a fairly old Ubuntu 6 machine
running httpd 2.0.55-4ubuntu4.1. I have since recreated it on my OS X
10.5 iMac using httpd 2.0.64 from MacPorts. I suppose I should also
point out that I am using PHP 5.2.9 to generate the chunked data. If
there's a way of doing chunked output using "plain" Apache let me know
and I can test that too if needed.

>> I'm not aware of any reason for this to behave specifically in this
>> way and clearly combining the entire chunk into a single SSL fragment
>> would provide a significant bandwidth saving for HTTP streaming
>> applications if not more mainstream ones.
>>
>> I've done a fair amount of poking through the httpd source today to
>> try and isolate this but this is my first foray into the depths of
>> httpd so I've not got far in the direction of a solution. I have
>> identified that this 'problem' is due to the way the chunk_filter
>> function adds buckets into the brigade which end up getting turned
>> into their own fragments by mod_ssl. Creating a new bucket which has
>> the extra data wrapped around it would presumably be far too
>> inefficient for a general solution. I was considering using the FLUSH
>> bucket type but am not fully aware of how it's used by the various
>> filters.
>
> It should not be necessary to have multiple buckets -- they should
> be written using a vector and not result in separate packets. However,
> this may be limited by the SSL library's write interface.
> FLUSH is the opposite of what you want.  We should either be doing
> the equivalent of a writev on SSL or add a buffering filter.
>
>> I'm not sure what the ideal way is to go about fixing this, or if it's
>> even in fact an actual source code level problem rather than a
>> configuration one, hence why this is posted to users for now.
>>
>> I can provide text dumps from tshark if that would be more
>> illuminating of the patterns I'm seeing.
>>
>> Alex Stapleton
>
> Platform details would be helpful -- you've narrowed the cause well
> enough that I doubt the text dumps would help.
>
> ....Roy
>

Re: mod_ssl and Transfer-Encoding: chunked wastes ~58 bytes per chunk.

[Alex Stapleton <al...@gmail.com>]

2009/8/21 Alex Stapleton <al...@gmail.com>:
> 2009/8/20 Roy T. Fielding <fi...@gbiv.com>:
>> On Aug 20, 2009, at 2:01 AM, Alex Stapleton wrote:
>>>
>>> 2009/8/19 Roy T. Fielding <fi...@gbiv.com>:
>>>>
>>>> On Aug 19, 2009, at 9:53 AM, Alex Stapleton wrote:
>>>>
>>>>> (This has been cross posted to users@. I apologise if this mail isn't
>>>>> relevant to the dev list.)
>>>>>
>>>>> First some background. We use Apache HTTPD 2.0 over a high-latency,
>>>>> high packet loss GPRS WAN. The cost per byte is tangible. We use SSL.
>>>>> We also use Transfer-Encoding: chunked sometimes. This is a machine
>>>>> monitoring application. We are using iframe streaming to push real
>>>>> time data to operators browsers.
>>>>>
>>>>> I have noticed after much faffing around with wireshark that httpd
>>>>> will transmit 3 Application Data fragments for each chunk in a chunked
>>>>> HTTP stream. The fragments are the HTTP chunk size, the chunk data,
>>>>> and then a CRLF. All fragments in an SSL session are rounded to the
>>>>> length of the ciphers block size. This results in the 4+2 bytes for
>>>>> the chunk frame ending up being 64 bytes of data over TCP. A waste of
>>>>> 58 bytes per chunk in this case.
>>>>
>>>> That's odd -- we don't do this with non-SSL writes (last I checked).
>>>> Perhaps we are relying on a TCP cork for the non-SSL case?
>>>> What is your operating system and platform?
>>>
>>> I initially discovered this issue on a fairly old Ubuntu 6 machine
>>> running httpd 2.0.55-4ubuntu4.1. I have since recreated it on my OS X
>>> 10.5 iMac using httpd 2.0.64 from MacPorts. I suppose I should also
>>> point out that I am using PHP 5.2.9 to generate the chunked data. If
>>> there's a way of doing chunked output using "plain" Apache let me know
>>> and I can test that too if needed.
>>
>> Er, have you tested it with 2.2.x?  The likelihood of us making any
>> non-security changes to the 2.0.x branch is extremely small.
>
> I have tested with 2.2.11 from MacPorts on my iMac and it also
> exhibits this behaviour. I'll try and do a 2.2.13 build to test with
> :)

Confirmed on 2.2.13.

>> Apache does chunked output by default if no content-length is
>> provided and the client says it is HTTP/1.1.
>>
>> ....Roy
>>
>>
>

Re: mod_ssl and Transfer-Encoding: chunked wastes ~58 bytes per chunk.

[Alex Stapleton <al...@gmail.com>]

2009/8/20 Roy T. Fielding <fi...@gbiv.com>:
> On Aug 20, 2009, at 2:01 AM, Alex Stapleton wrote:
>>
>> 2009/8/19 Roy T. Fielding <fi...@gbiv.com>:
>>>
>>> On Aug 19, 2009, at 9:53 AM, Alex Stapleton wrote:
>>>
>>>> (This has been cross posted to users@. I apologise if this mail isn't
>>>> relevant to the dev list.)
>>>>
>>>> First some background. We use Apache HTTPD 2.0 over a high-latency,
>>>> high packet loss GPRS WAN. The cost per byte is tangible. We use SSL.
>>>> We also use Transfer-Encoding: chunked sometimes. This is a machine
>>>> monitoring application. We are using iframe streaming to push real
>>>> time data to operators browsers.
>>>>
>>>> I have noticed after much faffing around with wireshark that httpd
>>>> will transmit 3 Application Data fragments for each chunk in a chunked
>>>> HTTP stream. The fragments are the HTTP chunk size, the chunk data,
>>>> and then a CRLF. All fragments in an SSL session are rounded to the
>>>> length of the ciphers block size. This results in the 4+2 bytes for
>>>> the chunk frame ending up being 64 bytes of data over TCP. A waste of
>>>> 58 bytes per chunk in this case.
>>>
>>> That's odd -- we don't do this with non-SSL writes (last I checked).
>>> Perhaps we are relying on a TCP cork for the non-SSL case?
>>> What is your operating system and platform?
>>
>> I initially discovered this issue on a fairly old Ubuntu 6 machine
>> running httpd 2.0.55-4ubuntu4.1. I have since recreated it on my OS X
>> 10.5 iMac using httpd 2.0.64 from MacPorts. I suppose I should also
>> point out that I am using PHP 5.2.9 to generate the chunked data. If
>> there's a way of doing chunked output using "plain" Apache let me know
>> and I can test that too if needed.
>
> Er, have you tested it with 2.2.x?  The likelihood of us making any
> non-security changes to the 2.0.x branch is extremely small.

I have tested with 2.2.11 from MacPorts on my iMac and it also
exhibits this behaviour. I'll try and do a 2.2.13 build to test with
:)

> Apache does chunked output by default if no content-length is
> provided and the client says it is HTTP/1.1.
>
> ....Roy
>
>

Re: mod_ssl and Transfer-Encoding: chunked wastes ~58 bytes per chunk.

["William A. Rowe, Jr." <wr...@rowe-clan.net>]

Roy T. Fielding wrote:
> 
> Apache does chunked output by default if no content-length is
> provided and the client says it is HTTP/1.1.

And further processes chunked input; it's not up to PHP or other
user agents to process this further, it's on the webserver to do so.

If you want to [de]chunk your own data, use mod_proxy_http.

Re: mod_ssl and Transfer-Encoding: chunked wastes ~58 bytes per chunk.

["Roy T. Fielding" <fi...@gbiv.com>]

On Aug 20, 2009, at 2:01 AM, Alex Stapleton wrote:
> 2009/8/19 Roy T. Fielding <fi...@gbiv.com>:
>> On Aug 19, 2009, at 9:53 AM, Alex Stapleton wrote:
>>
>>> (This has been cross posted to users@. I apologise if this mail  
>>> isn't
>>> relevant to the dev list.)
>>>
>>> First some background. We use Apache HTTPD 2.0 over a high-latency,
>>> high packet loss GPRS WAN. The cost per byte is tangible. We use  
>>> SSL.
>>> We also use Transfer-Encoding: chunked sometimes. This is a machine
>>> monitoring application. We are using iframe streaming to push real
>>> time data to operators browsers.
>>>
>>> I have noticed after much faffing around with wireshark that httpd
>>> will transmit 3 Application Data fragments for each chunk in a  
>>> chunked
>>> HTTP stream. The fragments are the HTTP chunk size, the chunk data,
>>> and then a CRLF. All fragments in an SSL session are rounded to the
>>> length of the ciphers block size. This results in the 4+2 bytes for
>>> the chunk frame ending up being 64 bytes of data over TCP. A  
>>> waste of
>>> 58 bytes per chunk in this case.
>>
>> That's odd -- we don't do this with non-SSL writes (last I checked).
>> Perhaps we are relying on a TCP cork for the non-SSL case?
>> What is your operating system and platform?
>
> I initially discovered this issue on a fairly old Ubuntu 6 machine
> running httpd 2.0.55-4ubuntu4.1. I have since recreated it on my OS X
> 10.5 iMac using httpd 2.0.64 from MacPorts. I suppose I should also
> point out that I am using PHP 5.2.9 to generate the chunked data. If
> there's a way of doing chunked output using "plain" Apache let me know
> and I can test that too if needed.

Er, have you tested it with 2.2.x?  The likelihood of us making any
non-security changes to the 2.0.x branch is extremely small.

Apache does chunked output by default if no content-length is
provided and the client says it is HTTP/1.1.

....Roy