Problem with cache+proxy and multiple brigade

[Matthieu Estrade <es...@ifrance.com>]

Hi,

I found a problem with mod_cache and mod_proxy...

when cache is storing the data + headers in memory, it doesn't care 
about Transfert-Encoding...
So if reverse proxy receive a response by multiple brigade, with the 
header Transfert-Encoding=chunked, mod_cache store it in mobj->header_out.

When cache_out serve the document from cache, it setup the 
content-length with mobj->m_len which is the len of the file stored in 
mobj->m
So if we had before the Transfer-Encoding set up, the response contains 
now these 2 headers:

Transfer-Encoding: chunked\r\n
Content-Length: 12457 \r\n

and in case of this response, the browser refuse to display the document 
(html, gif, css...etc)
I inserted these 2 lines before copying the headers (see patch) to 
delete the Transfer-Encoding in case reverse proxy receive it.

and now, the browser receive only content length and can display the 
document.
I am not sure my dirty patch is  good because i don't know all what 
Transfer-Encoding can take as value, and all impact of this header.

regards,

Estrade Matthieu

Re: Problem with cache+proxy and multiple brigade

[Graham Leggett <mi...@sharp.fm>]

Matthieu Estrade wrote:

> I can't see something in chunk_filter deleting the Transfer-Encoding 
> header if present and not chunked data...
> maybe i am in the wrong part of code. is there some dechunk_filter ?

There is a dechunk filter, which as I understand was merged into either 
CORE or CORE_IN (if very rusty memory serves).

Don't confuse chunking with dechunking, they do totally opposite things.

> but i don't understand why mod_cache should store this header, because 
> when it will serve the document from cache, i don't see why it should 
> take care about how proxy received the data because it store all the 
> data in the same brigade... cache->saved_brigade.
> So if we delete the header each time we serve from cache, it will take 
> more time than if we delete it before storing it in the cache_headers_out.

This is because chunking is something that only a chunking/dechunking 
filter should worry about.

Unfortunately there seems to be a significant number of places in the 
code where parts of Apache are fiddling with things that should never be 
fiddled with in that part of Apache. Things like proxy fiddling with 
content_length, etc.

The solution to all of this is to ensure the code only touches what it 
should touch. This is what the filter stacks are for.

Regards,
Graham
-- 
-----------------------------------------
minfrin@sharp.fm		"There's a moon
					over Bourbon Street
						tonight..."

Re: Problem with cache+proxy and multiple brigade

[Matthieu Estrade <me...@axiliance.com>]

Graham Leggett wrote:

> Matthieu Estrade wrote:
>
>> I found a problem with mod_cache and mod_proxy...
>>
>> when cache is storing the data + headers in memory, it doesn't care 
>> about Transfert-Encoding...
>
>
> AFAIK proxy should always dechunck any chunked data it receives before 
> passing it up the stack. The caching stuff should never see chunked data.
>
> If cache does, it could be because someone is putting the cache filter 
> in the wrong place in the stack, ie before the dechunk filter.


I can't see something in chunk_filter deleting the Transfer-Encoding 
header if present and not chunked data...
maybe i am in the wrong part of code. is there some dechunk_filter ?

but i don't understand why mod_cache should store this header, because 
when it will serve the document from cache, i don't see why it should 
take care about how proxy received the data because it store all the 
data in the same brigade... cache->saved_brigade.
So if we delete the header each time we serve from cache, it will take 
more time than if we delete it before storing it in the cache_headers_out.

>
>> +    /* Delete Transfer-Encoding if present, coming from reverse 
>> proxy */
>> +    +    if (apr_table_get(r->headers_out,"Transfer-Encoding")){
>> +        apr_table_unset(r->headers_out,"Transfer-Encoding");
>> +    }
>> + 
>
>
> Just arbly deleting the transfer encoding will definitely not work. 
> You need to make sure the content is dechunked as well. Only the 
> dechunk filter should remove the transfer encoding header. 

i will debug to see when cache_in and chunk_filter are called...

>
>
> Regards,
> Graham

Re: Problem with cache+proxy and multiple brigade

[Graham Leggett <mi...@sharp.fm>]

Matthieu Estrade wrote:

> I found a problem with mod_cache and mod_proxy...
> 
> when cache is storing the data + headers in memory, it doesn't care 
> about Transfert-Encoding...

AFAIK proxy should always dechunck any chunked data it receives before 
passing it up the stack. The caching stuff should never see chunked data.

If cache does, it could be because someone is putting the cache filter 
in the wrong place in the stack, ie before the dechunk filter.

> +    /* Delete Transfer-Encoding if present, coming from reverse proxy */
> +    
> +    if (apr_table_get(r->headers_out,"Transfer-Encoding")){
> +    	apr_table_unset(r->headers_out,"Transfer-Encoding");
> +    }
> + 

Just arbly deleting the transfer encoding will definitely not work. You 
need to make sure the content is dechunked as well. Only the dechunk 
filter should remove the transfer encoding header.

Regards,
Graham
-- 
-----------------------------------------
minfrin@sharp.fm		"There's a moon
					over Bourbon Street
						tonight..."