You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@httpd.apache.org by Alex Stapleton <al...@prol.etari.at> on 2009/08/18 22:26:24 UTC
[users@httpd] mod_ssl and Transfer-Encoding: chunked wastes ~58 bytes per chunk.
First some background. We use Apache HTTPD 2.0 over a high-latency,
high packet loss GPRS WAN. The cost per byte is tangible. We use SSL.
We also use Transfer-Encoding: chunked sometimes. This is a machine
monitoring application. We are using iframe streaming to push real
time data to operators browsers.
I have noticed after much faffing around with wireshark that httpd
will transmit 3 Application Data fragments for each chunk in a chunked
HTTP stream. The fragments are the HTTP chunk size, the chunk data,
and then a CRLF. All fragments in an SSL session are rounded to the
length of the ciphers block size. This results in the 4+2 bytes for
the chunk frame ending up being 64 bytes of data over TCP. A waste of
58 bytes per chunk in this case.
I'm not aware of any reason for this to behave specifically in this
way and clearly combining the entire chunk into a single SSL fragment
would provide a significant bandwidth saving for HTTP streaming
applications if not more mainstream ones.
I've done a fair amount of poking through the httpd source today to
try and isolate this but this is my first foray into the depths of
httpd so I've not got far in the direction of a solution. I have
identified that this 'problem' is due to the way the chunk_filter
function adds buckets into the brigade which end up getting turned
into their own fragments by mod_ssl. Creating a new bucket which has
the extra data wrapped around it would presumably be far too
inefficient for a general solution. I was considering using the FLUSH
bucket type but am not fully aware of how it's used by the various
filters.
I'm not sure what the ideal way is to go about fixing this, or if it's
even in fact an actual source code level problem rather than a
configuration one, hence why this is posted to users for now.
I can provide text dumps from tshark if that would be more
illuminating of the patterns I'm seeing.
Alex Stapleton
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
RE: mod_ssl and Transfer-Encoding: chunked wastes ~58 bytes perchunk.
Posted by "Plüm, Rüdiger, VF-Group" <ru...@vodafone.com>.
> -----Original Message-----
> From: Joe Orton
>
> Interesting observation.
>
> It would not be correct to fix this by adding buffering in the chunk
> filter. For a plain HTTP connection, any buffering/coalescing of
> packets is already done as necessary by the core output filter.
> Typically, a (chunk-size, data, crlf) brigade can get sent using
> writev() without requiring any copying.
>
> Translating many small buckets into many less-small SSL
> app-data records
> is certainly inefficient - and that's a property of SSL, so,
> I think it
> would be correct to fix this by adding some buffering in
> mod_ssl on the
> "plaintext" side of the output filter, i.e. in
> ssl_io_filter_output and
> ssl_filter_write.
>
> Any thoughts from dev@?
Sounds reasonable.
Regards
Rüdiger
[users@httpd] Re: mod_ssl and Transfer-Encoding: chunked wastes ~58 bytes per
chunk.
Posted by Joe Orton <jo...@redhat.com>.
CC'ing dev@.
On Tue, Aug 18, 2009 at 09:26:24PM +0100, Alex Stapleton wrote:
> First some background. We use Apache HTTPD 2.0 over a high-latency,
> high packet loss GPRS WAN. The cost per byte is tangible. We use SSL.
> We also use Transfer-Encoding: chunked sometimes. This is a machine
> monitoring application. We are using iframe streaming to push real
> time data to operators browsers.
>
> I have noticed after much faffing around with wireshark that httpd
> will transmit 3 Application Data fragments for each chunk in a chunked
> HTTP stream. The fragments are the HTTP chunk size, the chunk data,
> and then a CRLF. All fragments in an SSL session are rounded to the
> length of the ciphers block size. This results in the 4+2 bytes for
> the chunk frame ending up being 64 bytes of data over TCP. A waste of
> 58 bytes per chunk in this case.
Interesting observation.
It would not be correct to fix this by adding buffering in the chunk
filter. For a plain HTTP connection, any buffering/coalescing of
packets is already done as necessary by the core output filter.
Typically, a (chunk-size, data, crlf) brigade can get sent using
writev() without requiring any copying.
Translating many small buckets into many less-small SSL app-data records
is certainly inefficient - and that's a property of SSL, so, I think it
would be correct to fix this by adding some buffering in mod_ssl on the
"plaintext" side of the output filter, i.e. in ssl_io_filter_output and
ssl_filter_write.
Any thoughts from dev@?
Regards, Joe
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: mod_ssl and Transfer-Encoding: chunked wastes ~58 bytes per
chunk.
Posted by Joe Orton <jo...@redhat.com>.
CC'ing dev@.
On Tue, Aug 18, 2009 at 09:26:24PM +0100, Alex Stapleton wrote:
> First some background. We use Apache HTTPD 2.0 over a high-latency,
> high packet loss GPRS WAN. The cost per byte is tangible. We use SSL.
> We also use Transfer-Encoding: chunked sometimes. This is a machine
> monitoring application. We are using iframe streaming to push real
> time data to operators browsers.
>
> I have noticed after much faffing around with wireshark that httpd
> will transmit 3 Application Data fragments for each chunk in a chunked
> HTTP stream. The fragments are the HTTP chunk size, the chunk data,
> and then a CRLF. All fragments in an SSL session are rounded to the
> length of the ciphers block size. This results in the 4+2 bytes for
> the chunk frame ending up being 64 bytes of data over TCP. A waste of
> 58 bytes per chunk in this case.
Interesting observation.
It would not be correct to fix this by adding buffering in the chunk
filter. For a plain HTTP connection, any buffering/coalescing of
packets is already done as necessary by the core output filter.
Typically, a (chunk-size, data, crlf) brigade can get sent using
writev() without requiring any copying.
Translating many small buckets into many less-small SSL app-data records
is certainly inefficient - and that's a property of SSL, so, I think it
would be correct to fix this by adding some buffering in mod_ssl on the
"plaintext" side of the output filter, i.e. in ssl_io_filter_output and
ssl_filter_write.
Any thoughts from dev@?
Regards, Joe