You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@httpd.apache.org by sameer shah <sa...@gmail.com> on 2012/01/27 21:50:18 UTC
[users@httpd] How to make Apache mod_deflate and Transfer-encoding : Chunked work together?
I am trying to use the bigpipe concept on our website. That means trying to
send the response in chunks instead of sending it as a whole so that user
feels that page is fast. I am successful in doing that by using the
flushBuffer method on the response object in java. But now when I try to
compress the content with apache mod_deflate module, chunking is lost.
Here is the configuration from apache used to compress the content
**
Begin mod_deflate config
DeflateBufferSize 100
AddOutputFilterByType DEFLATE text/html text/plain text/xml text/javascript
BrowserMatch ^Mozilla/4 gzip-only-text/html
BrowserMatch ^Mozilla/4\.0[678] no-gzip
BrowserMatch \bMSIE !no-gzip !gzip-only-text/html
DeflateFilterNote Input input_info
DeflateFilterNote Output output_info
DeflateFilterNote Ratio ratio_info
LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate
CustomLog /var/log/httpd/deflate_log deflate
End mod_deflate config**
Here is the response header when the deflate is turned on in apache
Connection:Keep-Alive
*Content-Encoding:gzip*
Content-Length:7916
Content-Type:text/html; charset=UTF-8
Date:Fri, 27 Jan 2012 20:11:11 GMT
Keep-Alive:timeout=300, max=3997
Server:Apache
Vary:Accept-Encoding
Response header when the deflate is turned off in apache
Connection:Keep-Alive
Content-Type:text/html; charset=UTF-8
Date:Fri, 27 Jan 2012 20:21:14 GMT
Keep-Alive:timeout=300, max=3997
Server:Apache/2.2.3 (CentOS)
*Transfer-Encoding:chunked*
As you can see in above 2 headers chunking is working only if the
compression is turned off. I was searching on internet regarding this and
people were suggesting to decrease the *DeflateBufferSize*value. I
decreased the value to 100 bytes as you can see in my apache config but
that still didn't solve the problem. DeflateBufferSize set to 100 bytes
means that response is buffered in apache till 100 bytes are received and
then the it is compressed.
I was looking at the mod_gzip module which was bundled with the old apache
1.3 and that module has a following directive which allows chunked content
to be gzipped.
mod_gzip_dechunk Yes
Does anyone know of such directive in mod_deflate bundled with apache 2.x?
Or Does anyone know how to compress the chunked content?
Re: [users@httpd] How to make Apache mod_deflate and
Transfer-encoding : Chunked work together?
Posted by Matus UHLAR - fantomas <uh...@fantomas.sk>.
>On 2/1/2012 3:48 AM, Matus UHLAR - fantomas wrote:
>> Of course. But the original question was, why is chunking not used,
>> even when Content-Length was not sent? I don't know HTTP/1.1 enough
>> to answwer this question, do you?
On 01.02.12 12:50, William A. Rowe Jr. wrote:
>Yes; because the entire C-L is known and the overhead for C-L plus
>fragment header/trailer is longer than a simple C-L header.
This issue can also be handles by not sending the Content-Length and
using chunking.
>The original question was, "How do I force chunking". The answer
>is, you don't and can't expect to. Chunking is a hop-by-hop
>behavior over which you have no control by either endpoint of the
>intermediate servers' elections.
Maybe the original question should be rephrased to "How do I force
chunking when sending content from mod_gzip2", so we would not discuss
what chunking is for, if it collides with Content-Length and that it is
hop-by-hop header, and just search for answer for "why does mod_gzip
not send chunked responses, if it can".
--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
I just got lost in thought. It was unfamiliar territory.
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] How to make Apache mod_deflate and
Transfer-encoding : Chunked work together?
Posted by sameer shah <sa...@gmail.com>.
Hi William,
I have seen web sites forcing the chunked transfer-encoding as well as gzip
content encoding. One of the biggest example is *facebook*. And
*facebook*uses apache on linux.
Please refer this link on progressive rendering using chunked transfer
encoding.
http://www.phpied.com/progressive-rendering-via-multiple-flushes/
If you search "big pipe" on web you will find plenty of references of using
chunked transfer encoding.
Also apache does allow chunking if I don't enable mod_deflate module. Old
mod_gzip module which was packaged with old apache 1.3 had an directive
specifically for not dechunking.
*
*
*mod_gzip_dechunk No*
So status right now is if i don't use compression on apache response is
sent chunk by chunk.
Thanks
Sameer
On Wed, Feb 1, 2012 at 10:50 AM, William A. Rowe Jr. <wr...@rowe-clan.net>wrote:
> On 2/1/2012 3:48 AM, Matus UHLAR - fantomas wrote:
> >
> > Of course. But the original question was, why is chunking not used, even
> when
> > Content-Length was not sent? I don't know HTTP/1.1 enough to answwer
> this question, do you?
>
> Yes; because the entire C-L is known and the overhead for C-L plus
> fragment header/trailer is longer than a simple C-L header.
>
> The original question was, "How do I force chunking". The answer
> is, you don't and can't expect to. Chunking is a hop-by-hop
> behavior over which you have no control by either endpoint of the
> intermediate servers' elections.
>
> ---------------------------------------------------------------------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> " from the digest: users-digest-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
>
Re: [users@httpd] How to make Apache mod_deflate and Transfer-encoding
: Chunked work together?
Posted by "William A. Rowe Jr." <wr...@rowe-clan.net>.
On 2/1/2012 3:48 AM, Matus UHLAR - fantomas wrote:
>
> Of course. But the original question was, why is chunking not used, even when
> Content-Length was not sent? I don't know HTTP/1.1 enough to answwer this question, do you?
Yes; because the entire C-L is known and the overhead for C-L plus
fragment header/trailer is longer than a simple C-L header.
The original question was, "How do I force chunking". The answer
is, you don't and can't expect to. Chunking is a hop-by-hop
behavior over which you have no control by either endpoint of the
intermediate servers' elections.
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] How to make Apache mod_deflate and
Transfer-encoding : Chunked work together?
Posted by Matus UHLAR - fantomas <uh...@fantomas.sk>.
>>>>> On 1/27/2012 2:50 PM, sameer shah wrote:
>>>>>> As you can see in above 2 headers chunking is working only if
>>>>>> the compression is turned off.
>>>>
>>>> On 27.01.12 15:37, William A. Rowe Jr. wrote:
>>>>> Chunking is not a feature that the client gets to choose.
>>>>> Your client code is badly written if it is that fragile.
>On 1/31/2012 9:37 AM, Matus UHLAR - fantomas wrote:
>> IIUC, it's not badly written client code, but a want from webserver
>> admin, that seems not to be fullfillable with current
>> apache/mod_deflate...
On 31.01.12 11:07, William A. Rowe Jr. wrote:
>No. Protocols and specifications exist for a reason. Even *IF* you
>could force chunk behavior at the origin server, you would be incapable
>of producing the same results as the request travels through forward,
>reverse and transparent proxies outside of that administrators control.
Of course. But the original question was, why is chunking not used,
even when Content-Length was not sent? I don't know HTTP/1.1 enough to
answwer this question, do you?
--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
LSD will make your ECS screen display 16.7 million colors
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] How to make Apache mod_deflate and Transfer-encoding
: Chunked work together?
Posted by "William A. Rowe Jr." <wr...@rowe-clan.net>.
On 1/31/2012 9:37 AM, Matus UHLAR - fantomas wrote:
>>>> On 1/27/2012 2:50 PM, sameer shah wrote:
>>>>> As you can see in above 2 headers chunking is working only if the compression is
>>>>> turned off.
>>>
>>> On 27.01.12 15:37, William A. Rowe Jr. wrote:
>>>> Chunking is not a feature that the client gets to choose.
>>>> Your client code is badly written if it is that fragile.
>
> IIUC, it's not badly written client code, but a want from webserver admin, that seems not
> to be fullfillable with current apache/mod_deflate...
No. Protocols and specifications exist for a reason. Even *IF* you
could force chunk behavior at the origin server, you would be incapable
of producing the same results as the request travels through forward,
reverse and transparent proxies outside of that administrators control.
>>>> It has to be able to tolerate either of Content-Length or
>>>> Transfer-Encoding. You don't have command or control of the
>>>> mechanism used if you are an HTTP/1.1 client. It is always
>>>> the server's choice.
And tolerate disconnection to indicate end of response.
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] How to make Apache mod_deflate and
Transfer-encoding : Chunked work together?
Posted by Matus UHLAR - fantomas <uh...@fantomas.sk>.
>>> On 1/27/2012 2:50 PM, sameer shah wrote:
>>>> As you can see in above 2 headers chunking is working only if the
>>>> compression is turned off.
>>
>> On 27.01.12 15:37, William A. Rowe Jr. wrote:
>>> Chunking is not a feature that the client gets to choose.
>>> Your client code is badly written if it is that fragile.
IIUC, it's not badly written client code, but a want from webserver
admin, that seems not to be fullfillable with current
apache/mod_deflate...
>>> It has to be able to tolerate either of Content-Length or
>>> Transfer-Encoding. You don't have command or control of the
>>> mechanism used if you are an HTTP/1.1 client. It is always
>>> the server's choice.
>On 1/30/2012 8:30 AM, Matus UHLAR - fantomas wrote:
>> You can however send HTTP/1.0 request with which the server MUST NOT
>> send chunked response.
Looking at the original post this is just the opposite OP wants to
achieve.
On 30.01.12 13:06, William A. Rowe Jr. wrote:
>However, it need not send Content-Length either. If the C-L could
>not be determined, keepalives would not be honored, and you would be
>stuck in connection-per-request mode.
I agree, using http/1.0 is not what the OP wants.
--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
REALITY.SYS corrupted. Press any key to reboot Universe.
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] How to make Apache mod_deflate and Transfer-encoding
: Chunked work together?
Posted by "William A. Rowe Jr." <wr...@rowe-clan.net>.
On 1/30/2012 8:30 AM, Matus UHLAR - fantomas wrote:
>> On 1/27/2012 2:50 PM, sameer shah wrote:
>>> As you can see in above 2 headers chunking is working only if the compression is turned
>>> off.
>
> On 27.01.12 15:37, William A. Rowe Jr. wrote:
>> Chunking is not a feature that the client gets to choose.
>> Your client code is badly written if it is that fragile.
>>
>> It has to be able to tolerate either of Content-Length or
>> Transfer-Encoding. You don't have command or control of the
>> mechanism used if you are an HTTP/1.1 client. It is always
>> the server's choice.
>
> You can however send HTTP/1.0 request with which the server MUST NOT send chunked response.
However, it need not send Content-Length either. If the C-L could
not be determined, keepalives would not be honored, and you would be
stuck in connection-per-request mode.
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] How to make Apache mod_deflate and
Transfer-encoding : Chunked work together?
Posted by Matus UHLAR - fantomas <uh...@fantomas.sk>.
>On 1/27/2012 2:50 PM, sameer shah wrote:
>> As you can see in above 2 headers chunking is working only if the compression is turned
>> off.
On 27.01.12 15:37, William A. Rowe Jr. wrote:
>Chunking is not a feature that the client gets to choose.
>Your client code is badly written if it is that fragile.
>
>It has to be able to tolerate either of Content-Length or
>Transfer-Encoding. You don't have command or control of the
>mechanism used if you are an HTTP/1.1 client. It is always
>the server's choice.
You can however send HTTP/1.0 request with which the server MUST NOT
send chunked response.
--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Linux is like a teepee: no Windows, no Gates and an apache inside...
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] How to make Apache mod_deflate and Transfer-encoding
: Chunked work together?
Posted by "William A. Rowe Jr." <wr...@rowe-clan.net>.
On 1/27/2012 2:50 PM, sameer shah wrote:
> As you can see in above 2 headers chunking is working only if the compression is turned
> off.
Chunking is not a feature that the client gets to choose.
Your client code is badly written if it is that fragile.
It has to be able to tolerate either of Content-Length or
Transfer-Encoding. You don't have command or control of the
mechanism used if you are an HTTP/1.1 client. It is always
the server's choice.
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org