You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@trafficserver.apache.org by Jason Strongman <ja...@gmail.com> on 2014/09/30 01:49:13 UTC
regex_revalidate plugin and url defined in rule file
proxy mode - reverse proxy
related config option applied in my environment:
records.config
CONFIG proxy.config.url_remap.pristine_host_hdr INT 1
remap.config
map http://user.domain.com/ http://192.168.1.230/
when defining the re-validate urls, it appears the plugin regexp only
matches on the origin url
revalidate.config
# rule 1
# doesnt match original url -
http://user.domain.com/mymedia/title1/movie.mp4
user.domain.com/mymedia/title1/.* 1412033900
# rule 2
# matches original url -
http://user.domain.com/mymedia/title1/movie.mp4
192.168.1.230/mymedia/title1/.* 1412033900
So with rule 1 in place, requests for the original URL will not trigger a
re-validate.
With rule 2 in place, the original URL is re-validated against the origin.
I am thinking the origin URL is used due to how and what meta-data is
stored with object.
Or I could be way off base, and just misunderstand my observations.
Please advise.
Re: regex_revalidate plugin and url defined in rule file
Posted by Jason Strongman <ja...@gmail.com>.
OK thanks for the response!
i think i have enough of a grasp on things to get the rest of my answers by
reading source.
although you have cleared up 98% of my unknowns.
On Tue, Sep 30, 2014 at 8:26 AM, yu zou <zo...@yahoo-inc.com> wrote:
> Hi Jason,
> For question 1, currently, since regex_revalidate plugin checks the origin
> url, you must distinct the origin url after remap.
> For question 2, for the first time the original url matches the rule in
> regex_revalidate plugin, it will revalidate the cache, after that, if the
> rule which is related to origin url is not changed, the plugin will not
> revalidate the cache, just return.
>
> Thanks & Best wishes,
> Yu, Zou
>
>
> From: Jason Strongman <ja...@gmail.com>
> Reply-To: "users@trafficserver.apache.org" <users@trafficserver.apache.org
> >
> Date: Tuesday, September 30, 2014 at 9:10 PM
>
> To: "users@trafficserver.apache.org" <us...@trafficserver.apache.org>
> Subject: Re: regex_revalidate plugin and url defined in rule file
>
>
> since the plugin(in its current state) cant be called from a specific
> remap rule, how do you approach revalidation against a specific domain?
> especially when that specific domain shares the same origin address as
> another domain?
>
>
> records.config
>
> CONFIG proxy.config.url_remap.pristine_host_hdr INT 1
>
> remap.config
>
> map http://www.domain1.com/ http://192.168.1.230/
> map http://www.domain2.com/ http://192.168.1.230/
>
> revalidate.config
>
> 192.168.1.230/mymedia/title1/.*
> <rule-expire-time-in-seconds-since-epoch>
>
>
> With the above revalidate rule in place, requests for the below two URLs
> will trigger a revalidation.
>
> http://www.domain1.com/mymedia/title1/movie.mp4
> <http://user.domain.com/mymedia/title1/movie.mp4>
> http://www.domain2.com/mymedia/title1/movie.mp4
> <http://user.domain.com/mymedia/title1/movie.mp4>
>
> However, my intention is to only have www.domain1.com evaluated.
>
> My guess is that I would have to define my origins with a specific
> hostname instead. So the modified remap.config would look like,
>
> map http://www.domain1.com/ http://origin.domain1.com/
> map http://www.domain2.com/ http://origin.domain2.com/
>
>
>
> Another question..
>
> Since the revalidation plugin matches on the origin URL, and the only
> place the origin URL is stored is within the object meta-data.. does that
> mean that the plugin must inspect, and perform a cache lookup, for every
> single request?
>
> On Mon, Sep 29, 2014 at 9:30 PM, Jason Strongman <
> jasonstrongman2016@gmail.com> wrote:
>
>> thanks sir.. your explanation matches with my observations.
>>
>>
>>
>> On Mon, Sep 29, 2014 at 8:25 PM, yu zou <zo...@yahoo-inc.com> wrote:
>>
>>> And Jason, the regex_*revalidate plugin hooks at TS_HTTP_CACHE_*LOOKUP_COMPLETE_HOOK,
>>> it works after remap happens.
>>>
>>>
>>>
>>> From: Yu Zou <zo...@yahoo-inc.com>
>>> Reply-To: "users@trafficserver.apache.org" <
>>> users@trafficserver.apache.org>
>>> Date: Tuesday, September 30, 2014 at 9:16 AM
>>> To: "users@trafficserver.apache.org" <us...@trafficserver.apache.org>
>>> Subject: Re: regex_revalidate plugin and url defined in rule file
>>>
>>> HI Jason,
>>> regex_revalidate plugin matches the origin url and revalidates it.
>>> The scenario of it is used to revalidate the origin’s url which is stale
>>> in cache but is changed in origin server.
>>> So, in the rule file, please use the original’s url instead of user’s
>>> url.
>>>
>>> Thanks & Best wishes,
>>> Yu, Zou
>>>
>>>
>>> From: Jason Strongman <ja...@gmail.com>
>>> Reply-To: "users@trafficserver.apache.org" <
>>> users@trafficserver.apache.org>
>>> Date: Tuesday, September 30, 2014 at 7:49 AM
>>> To: "users@trafficserver.apache.org" <us...@trafficserver.apache.org>
>>> Subject: regex_revalidate plugin and url defined in rule file
>>>
>>>
>>>
>>> proxy mode - reverse proxy
>>>
>>> related config option applied in my environment:
>>>
>>> records.config
>>>
>>> CONFIG proxy.config.url_remap.pristine_host_hdr INT 1
>>>
>>> remap.config
>>>
>>> map http://user.domain.com/http://192.168.1.230/
>>>
>>>
>>> when defining the re-validate urls, it appears the plugin regexp only
>>> matches on the origin url
>>>
>>> revalidate.config
>>>
>>> # rule 1
>>> # doesnt match original url -
>>> http://user.domain.com/mymedia/title1/movie.mp4
>>>
>>> user.domain.com/mymedia/title1/.* 1412033900
>>>
>>> # rule 2
>>> # matches original url -
>>> http://user.domain.com/mymedia/title1/movie.mp4
>>>
>>> 192.168.1.230/mymedia/title1/.* 1412033900
>>>
>>>
>>>
>>> So with rule 1 in place, requests for the original URL will not trigger
>>> a re-validate.
>>> With rule 2 in place, the original URL is re-validated against the
>>> origin.
>>>
>>> I am thinking the origin URL is used due to how and what meta-data is
>>> stored with object.
>>>
>>> Or I could be way off base, and just misunderstand my observations.
>>>
>>>
>>> Please advise.
>>>
>>>
>>>
>>>
>>>
>>
>
Re: regex_revalidate plugin and url defined in rule file
Posted by yu zou <zo...@yahoo-inc.com>.
Hi Jason,
For question 1, currently, since regex_revalidate plugin checks the origin url, you must distinct the origin url after remap.
For question 2, for the first time the original url matches the rule in regex_revalidate plugin, it will revalidate the cache, after that, if the rule which is related to origin url is not changed, the plugin will not revalidate the cache, just return.
Thanks & Best wishes,
Yu, Zou
From: Jason Strongman <ja...@gmail.com>>
Reply-To: "users@trafficserver.apache.org<ma...@trafficserver.apache.org>" <us...@trafficserver.apache.org>>
Date: Tuesday, September 30, 2014 at 9:10 PM
To: "users@trafficserver.apache.org<ma...@trafficserver.apache.org>" <us...@trafficserver.apache.org>>
Subject: Re: regex_revalidate plugin and url defined in rule file
since the plugin(in its current state) cant be called from a specific remap rule, how do you approach revalidation against a specific domain? especially when that specific domain shares the same origin address as another domain?
records.config
CONFIG proxy.config.url_remap.pristine_host_hdr INT 1
remap.config
map http://www.domain1.com/ http://192.168.1.230/
map http://www.domain2.com/ http://192.168.1.230/
revalidate.config
192.168.1.230/mymedia/title1/.*<http://192.168.1.230/mymedia/title1/.*> <rule-expire-time-in-seconds-since-epoch>
With the above revalidate rule in place, requests for the below two URLs will trigger a revalidation.
http://www.domain1.com/mymedia/title1/movie.mp4<http://user.domain.com/mymedia/title1/movie.mp4>
http://www.domain2.com/mymedia/title1/movie.mp4<http://user.domain.com/mymedia/title1/movie.mp4>
However, my intention is to only have www.domain1.com<http://www.domain1.com> evaluated.
My guess is that I would have to define my origins with a specific hostname instead. So the modified remap.config would look like,
map http://www.domain1.com/ http://origin.domain1.com/
map http://www.domain2.com/ http://origin.domain2.com/
Another question..
Since the revalidation plugin matches on the origin URL, and the only place the origin URL is stored is within the object meta-data.. does that mean that the plugin must inspect, and perform a cache lookup, for every single request?
On Mon, Sep 29, 2014 at 9:30 PM, Jason Strongman <ja...@gmail.com>> wrote:
thanks sir.. your explanation matches with my observations.
On Mon, Sep 29, 2014 at 8:25 PM, yu zou <zo...@yahoo-inc.com>> wrote:
And Jason, the regex_revalidate plugin hooks at TS_HTTP_CACHE_LOOKUP_COMPLETE_HOOK, it works after remap happens.
From: Yu Zou <zo...@yahoo-inc.com>>
Reply-To: "users@trafficserver.apache.org<ma...@trafficserver.apache.org>" <us...@trafficserver.apache.org>>
Date: Tuesday, September 30, 2014 at 9:16 AM
To: "users@trafficserver.apache.org<ma...@trafficserver.apache.org>" <us...@trafficserver.apache.org>>
Subject: Re: regex_revalidate plugin and url defined in rule file
HI Jason,
regex_revalidate plugin matches the origin url and revalidates it.
The scenario of it is used to revalidate the origin’s url which is stale in cache but is changed in origin server.
So, in the rule file, please use the original’s url instead of user’s url.
Thanks & Best wishes,
Yu, Zou
From: Jason Strongman <ja...@gmail.com>>
Reply-To: "users@trafficserver.apache.org<ma...@trafficserver.apache.org>" <us...@trafficserver.apache.org>>
Date: Tuesday, September 30, 2014 at 7:49 AM
To: "users@trafficserver.apache.org<ma...@trafficserver.apache.org>" <us...@trafficserver.apache.org>>
Subject: regex_revalidate plugin and url defined in rule file
proxy mode - reverse proxy
related config option applied in my environment:
records.config
CONFIG proxy.config.url_remap.pristine_host_hdr INT 1
remap.config
map http://user.domain.com/http://192.168.1.230/
when defining the re-validate urls, it appears the plugin regexp only matches on the origin url
revalidate.config
# rule 1
# doesnt match original url - http://user.domain.com/mymedia/title1/movie.mp4
user.domain.com/mymedia/title1/.*<http://user.domain.com/mymedia/title1/.*> 1412033900
# rule 2
# matches original url - http://user.domain.com/mymedia/title1/movie.mp4
192.168.1.230/mymedia/title1/.*<http://192.168.1.230/mymedia/title1/.*> 1412033900
So with rule 1 in place, requests for the original URL will not trigger a re-validate.
With rule 2 in place, the original URL is re-validated against the origin.
I am thinking the origin URL is used due to how and what meta-data is stored with object.
Or I could be way off base, and just misunderstand my observations.
Please advise.
Re: regex_revalidate plugin and url defined in rule file
Posted by Jason Strongman <ja...@gmail.com>.
since the plugin(in its current state) cant be called from a specific
remap rule, how do you approach revalidation against a specific domain?
especially when that specific domain shares the same origin address as
another domain?
records.config
CONFIG proxy.config.url_remap.pristine_host_hdr INT 1
remap.config
map http://www.domain1.com/ http://192.168.1.230/
map http://www.domain2.com/ http://192.168.1.230/
revalidate.config
192.168.1.230/mymedia/title1/.*
<rule-expire-time-in-seconds-since-epoch>
With the above revalidate rule in place, requests for the below two URLs
will trigger a revalidation.
http://www.domain1.com/mymedia/title1/movie.mp4
<http://user.domain.com/mymedia/title1/movie.mp4>
http://www.domain2.com/mymedia/title1/movie.mp4
<http://user.domain.com/mymedia/title1/movie.mp4>
However, my intention is to only have www.domain1.com evaluated.
My guess is that I would have to define my origins with a specific hostname
instead. So the modified remap.config would look like,
map http://www.domain1.com/ http://origin.domain1.com/
map http://www.domain2.com/ http://origin.domain2.com/
Another question..
Since the revalidation plugin matches on the origin URL, and the only place
the origin URL is stored is within the object meta-data.. does that mean
that the plugin must inspect, and perform a cache lookup, for every single
request?
On Mon, Sep 29, 2014 at 9:30 PM, Jason Strongman <
jasonstrongman2016@gmail.com> wrote:
> thanks sir.. your explanation matches with my observations.
>
>
>
> On Mon, Sep 29, 2014 at 8:25 PM, yu zou <zo...@yahoo-inc.com> wrote:
>
>> And Jason, the regex_*revalidate plugin hooks at TS_HTTP_CACHE_*LOOKUP_COMPLETE_HOOK,
>> it works after remap happens.
>>
>>
>>
>> From: Yu Zou <zo...@yahoo-inc.com>
>> Reply-To: "users@trafficserver.apache.org" <
>> users@trafficserver.apache.org>
>> Date: Tuesday, September 30, 2014 at 9:16 AM
>> To: "users@trafficserver.apache.org" <us...@trafficserver.apache.org>
>> Subject: Re: regex_revalidate plugin and url defined in rule file
>>
>> HI Jason,
>> regex_revalidate plugin matches the origin url and revalidates it.
>> The scenario of it is used to revalidate the origin’s url which is stale
>> in cache but is changed in origin server.
>> So, in the rule file, please use the original’s url instead of user’s url.
>>
>> Thanks & Best wishes,
>> Yu, Zou
>>
>>
>> From: Jason Strongman <ja...@gmail.com>
>> Reply-To: "users@trafficserver.apache.org" <
>> users@trafficserver.apache.org>
>> Date: Tuesday, September 30, 2014 at 7:49 AM
>> To: "users@trafficserver.apache.org" <us...@trafficserver.apache.org>
>> Subject: regex_revalidate plugin and url defined in rule file
>>
>>
>>
>> proxy mode - reverse proxy
>>
>> related config option applied in my environment:
>>
>> records.config
>>
>> CONFIG proxy.config.url_remap.pristine_host_hdr INT 1
>>
>> remap.config
>>
>> map http://user.domain.com/ http://192.168.1.230/
>>
>>
>> when defining the re-validate urls, it appears the plugin regexp only
>> matches on the origin url
>>
>> revalidate.config
>>
>> # rule 1
>> # doesnt match original url -
>> http://user.domain.com/mymedia/title1/movie.mp4
>>
>> user.domain.com/mymedia/title1/.* 1412033900
>>
>> # rule 2
>> # matches original url -
>> http://user.domain.com/mymedia/title1/movie.mp4
>>
>> 192.168.1.230/mymedia/title1/.* 1412033900
>>
>>
>>
>> So with rule 1 in place, requests for the original URL will not trigger a
>> re-validate.
>> With rule 2 in place, the original URL is re-validated against the origin.
>>
>> I am thinking the origin URL is used due to how and what meta-data is
>> stored with object.
>>
>> Or I could be way off base, and just misunderstand my observations.
>>
>>
>> Please advise.
>>
>>
>>
>>
>>
>
Re: regex_revalidate plugin and url defined in rule file
Posted by Jason Strongman <ja...@gmail.com>.
thanks sir.. your explanation matches with my observations.
On Mon, Sep 29, 2014 at 8:25 PM, yu zou <zo...@yahoo-inc.com> wrote:
> And Jason, the regex_*revalidate plugin hooks at TS_HTTP_CACHE_*LOOKUP_COMPLETE_HOOK,
> it works after remap happens.
>
>
>
> From: Yu Zou <zo...@yahoo-inc.com>
> Reply-To: "users@trafficserver.apache.org" <users@trafficserver.apache.org
> >
> Date: Tuesday, September 30, 2014 at 9:16 AM
> To: "users@trafficserver.apache.org" <us...@trafficserver.apache.org>
> Subject: Re: regex_revalidate plugin and url defined in rule file
>
> HI Jason,
> regex_revalidate plugin matches the origin url and revalidates it.
> The scenario of it is used to revalidate the origin’s url which is stale
> in cache but is changed in origin server.
> So, in the rule file, please use the original’s url instead of user’s url.
>
> Thanks & Best wishes,
> Yu, Zou
>
>
> From: Jason Strongman <ja...@gmail.com>
> Reply-To: "users@trafficserver.apache.org" <users@trafficserver.apache.org
> >
> Date: Tuesday, September 30, 2014 at 7:49 AM
> To: "users@trafficserver.apache.org" <us...@trafficserver.apache.org>
> Subject: regex_revalidate plugin and url defined in rule file
>
>
>
> proxy mode - reverse proxy
>
> related config option applied in my environment:
>
> records.config
>
> CONFIG proxy.config.url_remap.pristine_host_hdr INT 1
>
> remap.config
>
> map http://user.domain.com/ http://192.168.1.230/
>
>
> when defining the re-validate urls, it appears the plugin regexp only
> matches on the origin url
>
> revalidate.config
>
> # rule 1
> # doesnt match original url -
> http://user.domain.com/mymedia/title1/movie.mp4
>
> user.domain.com/mymedia/title1/.* 1412033900
>
> # rule 2
> # matches original url -
> http://user.domain.com/mymedia/title1/movie.mp4
>
> 192.168.1.230/mymedia/title1/.* 1412033900
>
>
>
> So with rule 1 in place, requests for the original URL will not trigger a
> re-validate.
> With rule 2 in place, the original URL is re-validated against the origin.
>
> I am thinking the origin URL is used due to how and what meta-data is
> stored with object.
>
> Or I could be way off base, and just misunderstand my observations.
>
>
> Please advise.
>
>
>
>
>
Re: regex_revalidate plugin and url defined in rule file
Posted by yu zou <zo...@yahoo-inc.com>.
And Jason, the regex_revalidate plugin hooks at TS_HTTP_CACHE_LOOKUP_COMPLETE_HOOK, it works after remap happens.
From: Yu Zou <zo...@yahoo-inc.com>>
Reply-To: "users@trafficserver.apache.org<ma...@trafficserver.apache.org>" <us...@trafficserver.apache.org>>
Date: Tuesday, September 30, 2014 at 9:16 AM
To: "users@trafficserver.apache.org<ma...@trafficserver.apache.org>" <us...@trafficserver.apache.org>>
Subject: Re: regex_revalidate plugin and url defined in rule file
HI Jason,
regex_revalidate plugin matches the origin url and revalidates it.
The scenario of it is used to revalidate the origin’s url which is stale in cache but is changed in origin server.
So, in the rule file, please use the original’s url instead of user’s url.
Thanks & Best wishes,
Yu, Zou
From: Jason Strongman <ja...@gmail.com>>
Reply-To: "users@trafficserver.apache.org<ma...@trafficserver.apache.org>" <us...@trafficserver.apache.org>>
Date: Tuesday, September 30, 2014 at 7:49 AM
To: "users@trafficserver.apache.org<ma...@trafficserver.apache.org>" <us...@trafficserver.apache.org>>
Subject: regex_revalidate plugin and url defined in rule file
proxy mode - reverse proxy
related config option applied in my environment:
records.config
CONFIG proxy.config.url_remap.pristine_host_hdr INT 1
remap.config
map http://user.domain.com/ http://192.168.1.230/
when defining the re-validate urls, it appears the plugin regexp only matches on the origin url
revalidate.config
# rule 1
# doesnt match original url - http://user.domain.com/mymedia/title1/movie.mp4
user.domain.com/mymedia/title1/.*<http://user.domain.com/mymedia/title1/.*> 1412033900
# rule 2
# matches original url - http://user.domain.com/mymedia/title1/movie.mp4
192.168.1.230/mymedia/title1/.*<http://192.168.1.230/mymedia/title1/.*> 1412033900
So with rule 1 in place, requests for the original URL will not trigger a re-validate.
With rule 2 in place, the original URL is re-validated against the origin.
I am thinking the origin URL is used due to how and what meta-data is stored with object.
Or I could be way off base, and just misunderstand my observations.
Please advise.
Re: regex_revalidate plugin and url defined in rule file
Posted by yu zou <zo...@yahoo-inc.com>.
HI Jason,
regex_revalidate plugin matches the origin url and revalidates it.
The scenario of it is used to revalidate the origin’s url which is stale in cache but is changed in origin server.
So, in the rule file, please use the original’s url instead of user’s url.
Thanks & Best wishes,
Yu, Zou
From: Jason Strongman <ja...@gmail.com>>
Reply-To: "users@trafficserver.apache.org<ma...@trafficserver.apache.org>" <us...@trafficserver.apache.org>>
Date: Tuesday, September 30, 2014 at 7:49 AM
To: "users@trafficserver.apache.org<ma...@trafficserver.apache.org>" <us...@trafficserver.apache.org>>
Subject: regex_revalidate plugin and url defined in rule file
proxy mode - reverse proxy
related config option applied in my environment:
records.config
CONFIG proxy.config.url_remap.pristine_host_hdr INT 1
remap.config
map http://user.domain.com/ http://192.168.1.230/
when defining the re-validate urls, it appears the plugin regexp only matches on the origin url
revalidate.config
# rule 1
# doesnt match original url - http://user.domain.com/mymedia/title1/movie.mp4
user.domain.com/mymedia/title1/.*<http://user.domain.com/mymedia/title1/.*> 1412033900
# rule 2
# matches original url - http://user.domain.com/mymedia/title1/movie.mp4
192.168.1.230/mymedia/title1/.*<http://192.168.1.230/mymedia/title1/.*> 1412033900
So with rule 1 in place, requests for the original URL will not trigger a re-validate.
With rule 2 in place, the original URL is re-validated against the origin.
I am thinking the origin URL is used due to how and what meta-data is stored with object.
Or I could be way off base, and just misunderstand my observations.
Please advise.