You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Luca Toscano <to...@gmail.com> on 2017/10/08 09:59:05 UTC

mod_rewrite, vary headers and internal redirects

Hi everybody,

in https://bz.apache.org/bugzilla/show_bug.cgi?id=58231 I am trying to
figure out how to make Vary headers works for RewriteCond in the directory
context. From the docs:

http://httpd.apache.org/docs/current/mod/mod_rewrite.html

"If a HTTP header is used in a condition this header is added to the Vary
header of the response in case the condition evaluates to true for the
request. It is not added if the condition evaluates to false for the
request. Adding the HTTP header to the Vary header of the response is
needed for proper caching."

But this doesn't work yet in trunk within a Directory context, since IIUC
an internal redirect happens as consequence of a RewriteRule (following a
RewriteCond that evaluates to true) and all the response headers are
cleared out (including the Vary ones set). I added some attempts to fix the
issue to the bugzilla task, but the one that I like the most is having a
simple "keep-vary-header" note that is checked in
internal_internal_redirect().

Does this approach make sense? Is there any smarter way to do it?

Thanks a lot!

Luca

Re: mod_rewrite, vary headers and internal redirects

Posted by Luca Toscano <to...@gmail.com>.
Bring up the subject again. I do think that having <If> and <RewriteCond>
behaving in the same way when setting the Vary headers would be really
important, but I am not sure if this is a 2.5.x only change or if it is
worth to break the existing 2.4 behavior. Thoughts?

To summarize the current behavior:

1) <If HeaderFoo condition> sets Vary:HeaderFoo to the response regardless
of condition being true or false.
2) RewriteCond HeaderFoo condition sets Vary:HeaderFoo only when condition
is true (we state it very clearly in the docs).

Luca

2017-10-27 18:48 GMT+02:00 Luca Toscano <to...@gmail.com>:

> Took a look to the following docs:
>
> https://docs.trafficserver.apache.org/en/5.3.x/admin/
> http-proxy-caching.en.html
> https://varnish-cache.org/docs/5.1/users-guide/
> increasing-your-hitrate.html
> https://www.fastly.com/blog/best-practices-using-vary-header/
>
> The last one contains an interesting bit: "Vary to the Rescue" (the
> Vary:Accept-Encoding header example).
>
> It seems to me that almost all caches expand their caching keys with
> whatever header is added to "Vary". From the example that I've read, the
> only fact that a response for the URI /foo/bar may be different if
> Header:something is present or not in the request indicates a good use case
> for adding Vary:something to the response. For a caching proxy in fact the
> absence of a Header will become part of a key:
>
> URI x + no Header:something --> cached object 1
> URI x + Header:something=foo -> cached object 2
> URI x + Header:something=bar -> cached object 3
> etc..
>
> The fine tuning of this behavior to avoid cache pollution or a low hit
> rate can be controlled in RewriteConds via the NV (no-vary) configuration,
> so it shouldn't be a big issue. I am more convinced that the <If> behavior
> is the right one to follow..
>
> Luca
>
>
>
> 2017-10-25 23:08 GMT+02:00 Luca Toscano <to...@gmail.com>:
>
>> Hi Ruediger,
>>
>> the following patch (still to be carefully tested and/or improved) should
>> force RewriteCond to behave like an <If> block adding the Vary header
>> simply if the condition is evaluated (so header value present in the
>> request but not satisfying the condition or header completely absent):
>>
>> http://home.apache.org/~elukey/httpd-trunk-mod_rewrite-add_
>> vary_header_always.patch
>>
>> It seems to be a big change in behavior though, I'd be curious to find
>> out the motivations of the implementation choice at the time (will try to
>> dig a bit into svn history).
>>
>> Thanks!
>>
>> Luca
>>
>> 2017-10-23 8:43 GMT+02:00 Plüm, Rüdiger, Vodafone Group <
>> ruediger.pluem@vodafone.com>:
>>
>>> I would tend to say that the <If> code is correct and the RewriteCond
>>> code is wrong, because it doesn’t matter if the condition becomes true or
>>> false. The headers value has an influence on the result. Tricky question is
>>> what to do regarding Vary if the non-presence of a header has an influence.
>>>
>>>
>>>
>>> Regards
>>>
>>>
>>>
>>> Rüdiger
>>>
>>>
>>>
>>> *Von:* Luca Toscano [mailto:toscano.luca@gmail.com]
>>> *Gesendet:* Sonntag, 22. Oktober 2017 11:47
>>> *An:* Apache HTTP Server Development List <de...@httpd.apache.org>
>>> *Betreff:* Re: mod_rewrite, vary headers and internal redirects
>>>
>>>
>>>
>>> Hi everybody,
>>>
>>>
>>>
>>> 2017-10-09 13:46 GMT+02:00 Luca Toscano <to...@gmail.com>:
>>>
>>> Hi Yann,
>>>
>>>
>>>
>>> 2017-10-08 14:13 GMT+02:00 Yann Ylavic <yl...@gmail.com>:
>>>
>>> On Sun, Oct 8, 2017 at 2:03 PM, Yann Ylavic <yl...@gmail.com>
>>> wrote:
>>> > Hi Luca,
>>> >
>>> > On Sun, Oct 8, 2017 at 11:59 AM, Luca Toscano <to...@gmail.com>
>>> wrote:
>>> >>
>>> >> Does this approach make sense? Is there any smarter way to do it?
>>> >
>>> > I can't tell that I love the hack in internal redirects but looks like
>>> > a simple way to handle the case...
>>> > Nit: maybe a more descriptive name for the "keep-vary-header" note,
>>> > "redirect-keeps-vary"?
>>>
>>>
>>>
>>> +1
>>>
>>>
>>>
>>>
>>> But after all, if we reach an internal redirect with some Vary header
>>> already set, maybe we should never drop it, thus internal redirects
>>> should preserve Vary in any case...
>>>
>>>
>>>
>>> I'd prefer to limit the scope of the httpd configurations affected by
>>> this change to the minimum, but the change would probably look less hacky :)
>>>
>>>
>>>
>>>
>>>
>>> After https://svn.apache.org/r1811744 trunk should be inline with what
>>> the docs say, but I have another question now: a RewriteCond condition
>>> (containing something like HTTP:someheader) adds a Vary header to the
>>> response only if the condition evaluates to true, meanwhile a <If>
>>> condition adds the Vary header regardless. Is there any good motivation for
>>> this difference or they should be modified to be more consistent? The <If>
>>> block behavior seems to be more sound (after reading
>>> https://tools.ietf.org/html/rfc7231#section-7.1.4), but I'd like to
>>> hear more expert opinions :)
>>>
>>>
>>>
>>> Thanks!
>>>
>>>
>>>
>>> Luca
>>>
>>
>>
>

Re: mod_rewrite, vary headers and internal redirects

Posted by Luca Toscano <to...@gmail.com>.
Took a look to the following docs:

https://docs.trafficserver.apache.org/en/5.3.x/admin/http-proxy-caching.en.html
https://varnish-cache.org/docs/5.1/users-guide/increasing-your-hitrate.html
https://www.fastly.com/blog/best-practices-using-vary-header/

The last one contains an interesting bit: "Vary to the Rescue" (the
Vary:Accept-Encoding header example).

It seems to me that almost all caches expand their caching keys with
whatever header is added to "Vary". From the example that I've read, the
only fact that a response for the URI /foo/bar may be different if
Header:something is present or not in the request indicates a good use case
for adding Vary:something to the response. For a caching proxy in fact the
absence of a Header will become part of a key:

URI x + no Header:something --> cached object 1
URI x + Header:something=foo -> cached object 2
URI x + Header:something=bar -> cached object 3
etc..

The fine tuning of this behavior to avoid cache pollution or a low hit rate
can be controlled in RewriteConds via the NV (no-vary) configuration, so it
shouldn't be a big issue. I am more convinced that the <If> behavior is the
right one to follow..

Luca



2017-10-25 23:08 GMT+02:00 Luca Toscano <to...@gmail.com>:

> Hi Ruediger,
>
> the following patch (still to be carefully tested and/or improved) should
> force RewriteCond to behave like an <If> block adding the Vary header
> simply if the condition is evaluated (so header value present in the
> request but not satisfying the condition or header completely absent):
>
> http://home.apache.org/~elukey/httpd-trunk-mod_rewrite-add_vary_header_
> always.patch
>
> It seems to be a big change in behavior though, I'd be curious to find out
> the motivations of the implementation choice at the time (will try to dig a
> bit into svn history).
>
> Thanks!
>
> Luca
>
> 2017-10-23 8:43 GMT+02:00 Plüm, Rüdiger, Vodafone Group <
> ruediger.pluem@vodafone.com>:
>
>> I would tend to say that the <If> code is correct and the RewriteCond
>> code is wrong, because it doesn’t matter if the condition becomes true or
>> false. The headers value has an influence on the result. Tricky question is
>> what to do regarding Vary if the non-presence of a header has an influence.
>>
>>
>>
>> Regards
>>
>>
>>
>> Rüdiger
>>
>>
>>
>> *Von:* Luca Toscano [mailto:toscano.luca@gmail.com]
>> *Gesendet:* Sonntag, 22. Oktober 2017 11:47
>> *An:* Apache HTTP Server Development List <de...@httpd.apache.org>
>> *Betreff:* Re: mod_rewrite, vary headers and internal redirects
>>
>>
>>
>> Hi everybody,
>>
>>
>>
>> 2017-10-09 13:46 GMT+02:00 Luca Toscano <to...@gmail.com>:
>>
>> Hi Yann,
>>
>>
>>
>> 2017-10-08 14:13 GMT+02:00 Yann Ylavic <yl...@gmail.com>:
>>
>> On Sun, Oct 8, 2017 at 2:03 PM, Yann Ylavic <yl...@gmail.com> wrote:
>> > Hi Luca,
>> >
>> > On Sun, Oct 8, 2017 at 11:59 AM, Luca Toscano <to...@gmail.com>
>> wrote:
>> >>
>> >> Does this approach make sense? Is there any smarter way to do it?
>> >
>> > I can't tell that I love the hack in internal redirects but looks like
>> > a simple way to handle the case...
>> > Nit: maybe a more descriptive name for the "keep-vary-header" note,
>> > "redirect-keeps-vary"?
>>
>>
>>
>> +1
>>
>>
>>
>>
>> But after all, if we reach an internal redirect with some Vary header
>> already set, maybe we should never drop it, thus internal redirects
>> should preserve Vary in any case...
>>
>>
>>
>> I'd prefer to limit the scope of the httpd configurations affected by
>> this change to the minimum, but the change would probably look less hacky :)
>>
>>
>>
>>
>>
>> After https://svn.apache.org/r1811744 trunk should be inline with what
>> the docs say, but I have another question now: a RewriteCond condition
>> (containing something like HTTP:someheader) adds a Vary header to the
>> response only if the condition evaluates to true, meanwhile a <If>
>> condition adds the Vary header regardless. Is there any good motivation for
>> this difference or they should be modified to be more consistent? The <If>
>> block behavior seems to be more sound (after reading
>> https://tools.ietf.org/html/rfc7231#section-7.1.4), but I'd like to hear
>> more expert opinions :)
>>
>>
>>
>> Thanks!
>>
>>
>>
>> Luca
>>
>
>

Re: mod_rewrite, vary headers and internal redirects

Posted by Luca Toscano <to...@gmail.com>.
Hi Ruediger,

the following patch (still to be carefully tested and/or improved) should
force RewriteCond to behave like an <If> block adding the Vary header
simply if the condition is evaluated (so header value present in the
request but not satisfying the condition or header completely absent):

http://home.apache.org/~elukey/httpd-trunk-mod_rewrite-add_vary_header_always.patch

It seems to be a big change in behavior though, I'd be curious to find out
the motivations of the implementation choice at the time (will try to dig a
bit into svn history).

Thanks!

Luca

2017-10-23 8:43 GMT+02:00 Plüm, Rüdiger, Vodafone Group <
ruediger.pluem@vodafone.com>:

> I would tend to say that the <If> code is correct and the RewriteCond code
> is wrong, because it doesn’t matter if the condition becomes true or false.
> The headers value has an influence on the result. Tricky question is what
> to do regarding Vary if the non-presence of a header has an influence.
>
>
>
> Regards
>
>
>
> Rüdiger
>
>
>
> *Von:* Luca Toscano [mailto:toscano.luca@gmail.com]
> *Gesendet:* Sonntag, 22. Oktober 2017 11:47
> *An:* Apache HTTP Server Development List <de...@httpd.apache.org>
> *Betreff:* Re: mod_rewrite, vary headers and internal redirects
>
>
>
> Hi everybody,
>
>
>
> 2017-10-09 13:46 GMT+02:00 Luca Toscano <to...@gmail.com>:
>
> Hi Yann,
>
>
>
> 2017-10-08 14:13 GMT+02:00 Yann Ylavic <yl...@gmail.com>:
>
> On Sun, Oct 8, 2017 at 2:03 PM, Yann Ylavic <yl...@gmail.com> wrote:
> > Hi Luca,
> >
> > On Sun, Oct 8, 2017 at 11:59 AM, Luca Toscano <to...@gmail.com>
> wrote:
> >>
> >> Does this approach make sense? Is there any smarter way to do it?
> >
> > I can't tell that I love the hack in internal redirects but looks like
> > a simple way to handle the case...
> > Nit: maybe a more descriptive name for the "keep-vary-header" note,
> > "redirect-keeps-vary"?
>
>
>
> +1
>
>
>
>
> But after all, if we reach an internal redirect with some Vary header
> already set, maybe we should never drop it, thus internal redirects
> should preserve Vary in any case...
>
>
>
> I'd prefer to limit the scope of the httpd configurations affected by this
> change to the minimum, but the change would probably look less hacky :)
>
>
>
>
>
> After https://svn.apache.org/r1811744 trunk should be inline with what
> the docs say, but I have another question now: a RewriteCond condition
> (containing something like HTTP:someheader) adds a Vary header to the
> response only if the condition evaluates to true, meanwhile a <If>
> condition adds the Vary header regardless. Is there any good motivation for
> this difference or they should be modified to be more consistent? The <If>
> block behavior seems to be more sound (after reading
> https://tools.ietf.org/html/rfc7231#section-7.1.4), but I'd like to hear
> more expert opinions :)
>
>
>
> Thanks!
>
>
>
> Luca
>

AW: mod_rewrite, vary headers and internal redirects

Posted by Plüm, Rüdiger, Vodafone Group <ru...@vodafone.com>.
I would tend to say that the <If> code is correct and the RewriteCond code is wrong, because it doesn’t matter if the condition becomes true or false. The headers value has an influence on the result. Tricky question is what to do regarding Vary if the non-presence of a header has an influence.

Regards

Rüdiger

Von: Luca Toscano [mailto:toscano.luca@gmail.com]
Gesendet: Sonntag, 22. Oktober 2017 11:47
An: Apache HTTP Server Development List <de...@httpd.apache.org>
Betreff: Re: mod_rewrite, vary headers and internal redirects

Hi everybody,

2017-10-09 13:46 GMT+02:00 Luca Toscano <to...@gmail.com>>:
Hi Yann,

2017-10-08 14:13 GMT+02:00 Yann Ylavic <yl...@gmail.com>>:
On Sun, Oct 8, 2017 at 2:03 PM, Yann Ylavic <yl...@gmail.com>> wrote:
> Hi Luca,
>
> On Sun, Oct 8, 2017 at 11:59 AM, Luca Toscano <to...@gmail.com>> wrote:
>>
>> Does this approach make sense? Is there any smarter way to do it?
>
> I can't tell that I love the hack in internal redirects but looks like
> a simple way to handle the case...
> Nit: maybe a more descriptive name for the "keep-vary-header" note,
> "redirect-keeps-vary"?

+1


But after all, if we reach an internal redirect with some Vary header
already set, maybe we should never drop it, thus internal redirects
should preserve Vary in any case...

I'd prefer to limit the scope of the httpd configurations affected by this change to the minimum, but the change would probably look less hacky :)


After https://svn.apache.org/r1811744 trunk should be inline with what the docs say, but I have another question now: a RewriteCond condition (containing something like HTTP:someheader) adds a Vary header to the response only if the condition evaluates to true, meanwhile a <If> condition adds the Vary header regardless. Is there any good motivation for this difference or they should be modified to be more consistent? The <If> block behavior seems to be more sound (after reading https://tools.ietf.org/html/rfc7231#section-7.1.4), but I'd like to hear more expert opinions :)

Thanks!

Luca

Re: mod_rewrite, vary headers and internal redirects

Posted by Luca Toscano <to...@gmail.com>.
Hi everybody,

2017-10-09 13:46 GMT+02:00 Luca Toscano <to...@gmail.com>:

> Hi Yann,
>
> 2017-10-08 14:13 GMT+02:00 Yann Ylavic <yl...@gmail.com>:
>
>> On Sun, Oct 8, 2017 at 2:03 PM, Yann Ylavic <yl...@gmail.com> wrote:
>> > Hi Luca,
>> >
>> > On Sun, Oct 8, 2017 at 11:59 AM, Luca Toscano <to...@gmail.com>
>> wrote:
>> >>
>> >> Does this approach make sense? Is there any smarter way to do it?
>> >
>> > I can't tell that I love the hack in internal redirects but looks like
>> > a simple way to handle the case...
>> > Nit: maybe a more descriptive name for the "keep-vary-header" note,
>> > "redirect-keeps-vary"?
>>
>
> +1
>
>
>>
>> But after all, if we reach an internal redirect with some Vary header
>> already set, maybe we should never drop it, thus internal redirects
>> should preserve Vary in any case...
>>
>
> I'd prefer to limit the scope of the httpd configurations affected by this
> change to the minimum, but the change would probably look less hacky :)
>


After https://svn.apache.org/r1811744 trunk should be inline with what the
docs say, but I have another question now: a RewriteCond condition
(containing something like HTTP:someheader) adds a Vary header to the
response only if the condition evaluates to true, meanwhile a <If>
condition adds the Vary header regardless. Is there any good motivation for
this difference or they should be modified to be more consistent? The <If>
block behavior seems to be more sound (after reading
https://tools.ietf.org/html/rfc7231#section-7.1.4), but I'd like to hear
more expert opinions :)

Thanks!

Luca

Re: mod_rewrite, vary headers and internal redirects

Posted by Luca Toscano <to...@gmail.com>.
Hi Yann,

2017-10-08 14:13 GMT+02:00 Yann Ylavic <yl...@gmail.com>:

> On Sun, Oct 8, 2017 at 2:03 PM, Yann Ylavic <yl...@gmail.com> wrote:
> > Hi Luca,
> >
> > On Sun, Oct 8, 2017 at 11:59 AM, Luca Toscano <to...@gmail.com>
> wrote:
> >>
> >> Does this approach make sense? Is there any smarter way to do it?
> >
> > I can't tell that I love the hack in internal redirects but looks like
> > a simple way to handle the case...
> > Nit: maybe a more descriptive name for the "keep-vary-header" note,
> > "redirect-keeps-vary"?
>

+1


>
> But after all, if we reach an internal redirect with some Vary header
> already set, maybe we should never drop it, thus internal redirects
> should preserve Vary in any case...
>

I'd prefer to limit the scope of the httpd configurations affected by this
change to the minimum, but the change would probably look less hacky :)

Luca

Re: mod_rewrite, vary headers and internal redirects

Posted by Yann Ylavic <yl...@gmail.com>.
On Sun, Oct 8, 2017 at 2:03 PM, Yann Ylavic <yl...@gmail.com> wrote:
> Hi Luca,
>
> On Sun, Oct 8, 2017 at 11:59 AM, Luca Toscano <to...@gmail.com> wrote:
>>
>> Does this approach make sense? Is there any smarter way to do it?
>
> I can't tell that I love the hack in internal redirects but looks like
> a simple way to handle the case...
> Nit: maybe a more descriptive name for the "keep-vary-header" note,
> "redirect-keeps-vary"?

But after all, if we reach an internal redirect with some Vary header
already set, maybe we should never drop it, thus internal redirects
should preserve Vary in any case...

>
> Regards,
> Yann.

Re: mod_rewrite, vary headers and internal redirects

Posted by Yann Ylavic <yl...@gmail.com>.
Hi Luca,

On Sun, Oct 8, 2017 at 11:59 AM, Luca Toscano <to...@gmail.com> wrote:
>
> Does this approach make sense? Is there any smarter way to do it?

I can't tell that I love the hack in internal redirects but looks like
a simple way to handle the case...
Nit: maybe a more descriptive name for the "keep-vary-header" note,
"redirect-keeps-vary"?

Regards,
Yann.