You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@trafficcontrol.apache.org by Jeremy Mitchell <mi...@apache.org> on 2016/12/12 22:43:53 UTC
regex_revalidate.config
I've created an issue to solicit some feedback regarding the removal of
regex_revalidate.config from the file system. If you are not aware,
regex_revalidate.config is the ATS config file used to "invalidate
content". Storing regex_revalidate.config on the file system makes it
harder to set up a highly-available, redundant traffic ops.
https://issues.apache.org/jira/browse/TC-68
Thanks!
Re: regex_revalidate.config
Posted by Jeremy Mitchell <mi...@gmail.com>.
Ok, I also talked to the Comcast team that agreed this had to be done so
here is my PR:
https://github.com/apache/incubator-trafficcontrol/pull/200
^^ please comment on the PR if you see any problems
On Thu, Jan 19, 2017 at 6:19 AM, Steve Malenfant <sm...@gmail.com>
wrote:
> Although I've seen higher duration of requests creating issues with ORT
> updating all our servers in the past, I think this one is required to move
> to HA. I'd like to see what's going to be the impact in a big production
> environment. Other than that. +1
>
> On Tue, Jan 17, 2017 at 5:39 PM, Jeremy Mitchell <mi...@gmail.com>
> wrote:
>
> > Does anyone have a problem with me moving forward with
> > https://issues.apache.org/jira/browse/TC-68
> >
> > Yes, this will increase the amount of time it takes for ORT to fetch
> > regex_revalidate.config (see my curl durations in previous response) but,
> > hopefully, the PR submitted by @dg4prez (
> > https://github.com/apache/incubator-trafficcontrol/pull/141) plus a
> > caching
> > layer in front of TO will negate the additional latency.
> >
> > Why is this issue important? Without removing regex_revalidate.config
> from
> > the file system (it is currently stored
> > at public/Trafficserver-Snapshots/:cdn_name/regex_revalidate.config), it
> > makes it difficult to move to an HA TO (highly-available, redundant).
> >
> > Jeremy
> >
> >
> >
> > On Thu, Jan 5, 2017 at 4:42 PM, Jeremy Mitchell <mi...@gmail.com>
> > wrote:
> >
> > > derek gelinas is working on a PR that "scopes" these config files. i.e.
> > > some config files are at the server level, profile level or cdn level.
> > > regex_revalidate.config happens to be at the cdn level. Here is his PR
> > that
> > > I think will be targeted for 2.1.
> > >
> > > https://github.com/apache/incubator-trafficcontrol/pull/141
> > >
> > > what this means is that if a server needs to fetch
> > > regex_revalidate.config, it would fetch it like this
> > > to-domain.com/api/1.2/cdn/cdn1/configfiles/ats/regex_revalidate.config
> > > and if this file is served from a cache of its own, the request by X
> > number
> > > of servers to get the same file would take advantage of caching to
> reduce
> > > the delivery time. sooo.....maybe it's a second the first time and a
> few
> > ms
> > > each subsequent time...
> > >
> > > hope that made sense.
> > >
> > > jeremy
> > >
> > >
> > >
> > > On Tue, Jan 3, 2017 at 7:05 PM, Steve Malenfant <sm...@gmail.com>
> > > wrote:
> > >
> > >> Jeremy,
> > >>
> > >> I missed this email. Seems like close to a second is quite slow. Is
> the
> > >> regex_revalidate.config per server? It seems like it was per CDN
> before.
> > >>
> > >> On Tue, Dec 13, 2016 at 1:02 PM, Jeremy Mitchell <
> mitchell852@gmail.com
> > >
> > >> wrote:
> > >>
> > >> > Obviously, it is going to be slower to pull regex_revalidate.config
> > >> thru TO
> > >> > as opposed to pulling it off the file system. The question is...is
> > this
> > >> > acceptable?
> > >> >
> > >> > curling to
> > >> > https://to.domain.com/Trafficserver-Snapshots/cdn-
> > >> > name/regex_revalidate.config
> > >> >
> > >> > time_namelookup: 0.005
> > >> > time_connect: 0.058
> > >> > time_appconnect: 0.000
> > >> > time_pretransfer: 0.000
> > >> > time_redirect: 0.000
> > >> > time_starttransfer: 0.000
> > >> > ----------
> > >> > time_total: 0.063
> > >> >
> > >> > curling to
> > >> > https://to.domain.com/genfiles/view/server-host-
> > >> > name/regex_revalidate.config
> > >> >
> > >> > time_namelookup: 0.005
> > >> > time_connect: 0.062
> > >> > time_appconnect: 0.307
> > >> > time_pretransfer: 0.307
> > >> > time_redirect: 0.000
> > >> > time_starttransfer: 0.781
> > >> > ----------
> > >> > time_total: 0.781
> > >> >
> > >> > On Mon, Dec 12, 2016 at 9:03 PM, Steve Malenfant <
> > smalenfant@gmail.com>
> > >> > wrote:
> > >> >
> > >> > > I'm OK with this. I think most of the revalidate functionality is
> > not
> > >> > well
> > >> > > understood here... I guess I need to look into the API since we
> are
> > >> using
> > >> > > the UI to issue revalidate today.
> > >> > >
> > >> > > On Mon, Dec 12, 2016 at 5:43 PM, Jeremy Mitchell <
> > >> mitchell852@apache.org
> > >> > >
> > >> > > wrote:
> > >> > >
> > >> > > > I've created an issue to solicit some feedback regarding the
> > >> removal of
> > >> > > > regex_revalidate.config from the file system. If you are not
> > aware,
> > >> > > > regex_revalidate.config is the ATS config file used to
> "invalidate
> > >> > > > content". Storing regex_revalidate.config on the file system
> makes
> > >> it
> > >> > > > harder to set up a highly-available, redundant traffic ops.
> > >> > > >
> > >> > > > https://issues.apache.org/jira/browse/TC-68
> > >> > > >
> > >> > > > Thanks!
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> > >
> >
>
Re: regex_revalidate.config
Posted by Steve Malenfant <sm...@gmail.com>.
Although I've seen higher duration of requests creating issues with ORT
updating all our servers in the past, I think this one is required to move
to HA. I'd like to see what's going to be the impact in a big production
environment. Other than that. +1
On Tue, Jan 17, 2017 at 5:39 PM, Jeremy Mitchell <mi...@gmail.com>
wrote:
> Does anyone have a problem with me moving forward with
> https://issues.apache.org/jira/browse/TC-68
>
> Yes, this will increase the amount of time it takes for ORT to fetch
> regex_revalidate.config (see my curl durations in previous response) but,
> hopefully, the PR submitted by @dg4prez (
> https://github.com/apache/incubator-trafficcontrol/pull/141) plus a
> caching
> layer in front of TO will negate the additional latency.
>
> Why is this issue important? Without removing regex_revalidate.config from
> the file system (it is currently stored
> at public/Trafficserver-Snapshots/:cdn_name/regex_revalidate.config), it
> makes it difficult to move to an HA TO (highly-available, redundant).
>
> Jeremy
>
>
>
> On Thu, Jan 5, 2017 at 4:42 PM, Jeremy Mitchell <mi...@gmail.com>
> wrote:
>
> > derek gelinas is working on a PR that "scopes" these config files. i.e.
> > some config files are at the server level, profile level or cdn level.
> > regex_revalidate.config happens to be at the cdn level. Here is his PR
> that
> > I think will be targeted for 2.1.
> >
> > https://github.com/apache/incubator-trafficcontrol/pull/141
> >
> > what this means is that if a server needs to fetch
> > regex_revalidate.config, it would fetch it like this
> > to-domain.com/api/1.2/cdn/cdn1/configfiles/ats/regex_revalidate.config
> > and if this file is served from a cache of its own, the request by X
> number
> > of servers to get the same file would take advantage of caching to reduce
> > the delivery time. sooo.....maybe it's a second the first time and a few
> ms
> > each subsequent time...
> >
> > hope that made sense.
> >
> > jeremy
> >
> >
> >
> > On Tue, Jan 3, 2017 at 7:05 PM, Steve Malenfant <sm...@gmail.com>
> > wrote:
> >
> >> Jeremy,
> >>
> >> I missed this email. Seems like close to a second is quite slow. Is the
> >> regex_revalidate.config per server? It seems like it was per CDN before.
> >>
> >> On Tue, Dec 13, 2016 at 1:02 PM, Jeremy Mitchell <mitchell852@gmail.com
> >
> >> wrote:
> >>
> >> > Obviously, it is going to be slower to pull regex_revalidate.config
> >> thru TO
> >> > as opposed to pulling it off the file system. The question is...is
> this
> >> > acceptable?
> >> >
> >> > curling to
> >> > https://to.domain.com/Trafficserver-Snapshots/cdn-
> >> > name/regex_revalidate.config
> >> >
> >> > time_namelookup: 0.005
> >> > time_connect: 0.058
> >> > time_appconnect: 0.000
> >> > time_pretransfer: 0.000
> >> > time_redirect: 0.000
> >> > time_starttransfer: 0.000
> >> > ----------
> >> > time_total: 0.063
> >> >
> >> > curling to
> >> > https://to.domain.com/genfiles/view/server-host-
> >> > name/regex_revalidate.config
> >> >
> >> > time_namelookup: 0.005
> >> > time_connect: 0.062
> >> > time_appconnect: 0.307
> >> > time_pretransfer: 0.307
> >> > time_redirect: 0.000
> >> > time_starttransfer: 0.781
> >> > ----------
> >> > time_total: 0.781
> >> >
> >> > On Mon, Dec 12, 2016 at 9:03 PM, Steve Malenfant <
> smalenfant@gmail.com>
> >> > wrote:
> >> >
> >> > > I'm OK with this. I think most of the revalidate functionality is
> not
> >> > well
> >> > > understood here... I guess I need to look into the API since we are
> >> using
> >> > > the UI to issue revalidate today.
> >> > >
> >> > > On Mon, Dec 12, 2016 at 5:43 PM, Jeremy Mitchell <
> >> mitchell852@apache.org
> >> > >
> >> > > wrote:
> >> > >
> >> > > > I've created an issue to solicit some feedback regarding the
> >> removal of
> >> > > > regex_revalidate.config from the file system. If you are not
> aware,
> >> > > > regex_revalidate.config is the ATS config file used to "invalidate
> >> > > > content". Storing regex_revalidate.config on the file system makes
> >> it
> >> > > > harder to set up a highly-available, redundant traffic ops.
> >> > > >
> >> > > > https://issues.apache.org/jira/browse/TC-68
> >> > > >
> >> > > > Thanks!
> >> > > >
> >> > >
> >> >
> >>
> >
> >
>
Re: regex_revalidate.config
Posted by Jeremy Mitchell <mi...@gmail.com>.
Does anyone have a problem with me moving forward with
https://issues.apache.org/jira/browse/TC-68
Yes, this will increase the amount of time it takes for ORT to fetch
regex_revalidate.config (see my curl durations in previous response) but,
hopefully, the PR submitted by @dg4prez (
https://github.com/apache/incubator-trafficcontrol/pull/141) plus a caching
layer in front of TO will negate the additional latency.
Why is this issue important? Without removing regex_revalidate.config from
the file system (it is currently stored
at public/Trafficserver-Snapshots/:cdn_name/regex_revalidate.config), it
makes it difficult to move to an HA TO (highly-available, redundant).
Jeremy
On Thu, Jan 5, 2017 at 4:42 PM, Jeremy Mitchell <mi...@gmail.com>
wrote:
> derek gelinas is working on a PR that "scopes" these config files. i.e.
> some config files are at the server level, profile level or cdn level.
> regex_revalidate.config happens to be at the cdn level. Here is his PR that
> I think will be targeted for 2.1.
>
> https://github.com/apache/incubator-trafficcontrol/pull/141
>
> what this means is that if a server needs to fetch
> regex_revalidate.config, it would fetch it like this
> to-domain.com/api/1.2/cdn/cdn1/configfiles/ats/regex_revalidate.config
> and if this file is served from a cache of its own, the request by X number
> of servers to get the same file would take advantage of caching to reduce
> the delivery time. sooo.....maybe it's a second the first time and a few ms
> each subsequent time...
>
> hope that made sense.
>
> jeremy
>
>
>
> On Tue, Jan 3, 2017 at 7:05 PM, Steve Malenfant <sm...@gmail.com>
> wrote:
>
>> Jeremy,
>>
>> I missed this email. Seems like close to a second is quite slow. Is the
>> regex_revalidate.config per server? It seems like it was per CDN before.
>>
>> On Tue, Dec 13, 2016 at 1:02 PM, Jeremy Mitchell <mi...@gmail.com>
>> wrote:
>>
>> > Obviously, it is going to be slower to pull regex_revalidate.config
>> thru TO
>> > as opposed to pulling it off the file system. The question is...is this
>> > acceptable?
>> >
>> > curling to
>> > https://to.domain.com/Trafficserver-Snapshots/cdn-
>> > name/regex_revalidate.config
>> >
>> > time_namelookup: 0.005
>> > time_connect: 0.058
>> > time_appconnect: 0.000
>> > time_pretransfer: 0.000
>> > time_redirect: 0.000
>> > time_starttransfer: 0.000
>> > ----------
>> > time_total: 0.063
>> >
>> > curling to
>> > https://to.domain.com/genfiles/view/server-host-
>> > name/regex_revalidate.config
>> >
>> > time_namelookup: 0.005
>> > time_connect: 0.062
>> > time_appconnect: 0.307
>> > time_pretransfer: 0.307
>> > time_redirect: 0.000
>> > time_starttransfer: 0.781
>> > ----------
>> > time_total: 0.781
>> >
>> > On Mon, Dec 12, 2016 at 9:03 PM, Steve Malenfant <sm...@gmail.com>
>> > wrote:
>> >
>> > > I'm OK with this. I think most of the revalidate functionality is not
>> > well
>> > > understood here... I guess I need to look into the API since we are
>> using
>> > > the UI to issue revalidate today.
>> > >
>> > > On Mon, Dec 12, 2016 at 5:43 PM, Jeremy Mitchell <
>> mitchell852@apache.org
>> > >
>> > > wrote:
>> > >
>> > > > I've created an issue to solicit some feedback regarding the
>> removal of
>> > > > regex_revalidate.config from the file system. If you are not aware,
>> > > > regex_revalidate.config is the ATS config file used to "invalidate
>> > > > content". Storing regex_revalidate.config on the file system makes
>> it
>> > > > harder to set up a highly-available, redundant traffic ops.
>> > > >
>> > > > https://issues.apache.org/jira/browse/TC-68
>> > > >
>> > > > Thanks!
>> > > >
>> > >
>> >
>>
>
>
Re: regex_revalidate.config
Posted by Jeremy Mitchell <mi...@gmail.com>.
derek gelinas is working on a PR that "scopes" these config files. i.e.
some config files are at the server level, profile level or cdn level.
regex_revalidate.config happens to be at the cdn level. Here is his PR that
I think will be targeted for 2.1.
https://github.com/apache/incubator-trafficcontrol/pull/141
what this means is that if a server needs to fetch regex_revalidate.config,
it would fetch it like this
to-domain.com/api/1.2/cdn/cdn1/configfiles/ats/regex_revalidate.config and
if this file is served from a cache of its own, the request by X number of
servers to get the same file would take advantage of caching to reduce the
delivery time. sooo.....maybe it's a second the first time and a few ms
each subsequent time...
hope that made sense.
jeremy
On Tue, Jan 3, 2017 at 7:05 PM, Steve Malenfant <sm...@gmail.com>
wrote:
> Jeremy,
>
> I missed this email. Seems like close to a second is quite slow. Is the
> regex_revalidate.config per server? It seems like it was per CDN before.
>
> On Tue, Dec 13, 2016 at 1:02 PM, Jeremy Mitchell <mi...@gmail.com>
> wrote:
>
> > Obviously, it is going to be slower to pull regex_revalidate.config thru
> TO
> > as opposed to pulling it off the file system. The question is...is this
> > acceptable?
> >
> > curling to
> > https://to.domain.com/Trafficserver-Snapshots/cdn-
> > name/regex_revalidate.config
> >
> > time_namelookup: 0.005
> > time_connect: 0.058
> > time_appconnect: 0.000
> > time_pretransfer: 0.000
> > time_redirect: 0.000
> > time_starttransfer: 0.000
> > ----------
> > time_total: 0.063
> >
> > curling to
> > https://to.domain.com/genfiles/view/server-host-
> > name/regex_revalidate.config
> >
> > time_namelookup: 0.005
> > time_connect: 0.062
> > time_appconnect: 0.307
> > time_pretransfer: 0.307
> > time_redirect: 0.000
> > time_starttransfer: 0.781
> > ----------
> > time_total: 0.781
> >
> > On Mon, Dec 12, 2016 at 9:03 PM, Steve Malenfant <sm...@gmail.com>
> > wrote:
> >
> > > I'm OK with this. I think most of the revalidate functionality is not
> > well
> > > understood here... I guess I need to look into the API since we are
> using
> > > the UI to issue revalidate today.
> > >
> > > On Mon, Dec 12, 2016 at 5:43 PM, Jeremy Mitchell <
> mitchell852@apache.org
> > >
> > > wrote:
> > >
> > > > I've created an issue to solicit some feedback regarding the removal
> of
> > > > regex_revalidate.config from the file system. If you are not aware,
> > > > regex_revalidate.config is the ATS config file used to "invalidate
> > > > content". Storing regex_revalidate.config on the file system makes it
> > > > harder to set up a highly-available, redundant traffic ops.
> > > >
> > > > https://issues.apache.org/jira/browse/TC-68
> > > >
> > > > Thanks!
> > > >
> > >
> >
>
Re: regex_revalidate.config
Posted by Steve Malenfant <sm...@gmail.com>.
Jeremy,
I missed this email. Seems like close to a second is quite slow. Is the
regex_revalidate.config per server? It seems like it was per CDN before.
On Tue, Dec 13, 2016 at 1:02 PM, Jeremy Mitchell <mi...@gmail.com>
wrote:
> Obviously, it is going to be slower to pull regex_revalidate.config thru TO
> as opposed to pulling it off the file system. The question is...is this
> acceptable?
>
> curling to
> https://to.domain.com/Trafficserver-Snapshots/cdn-
> name/regex_revalidate.config
>
> time_namelookup: 0.005
> time_connect: 0.058
> time_appconnect: 0.000
> time_pretransfer: 0.000
> time_redirect: 0.000
> time_starttransfer: 0.000
> ----------
> time_total: 0.063
>
> curling to
> https://to.domain.com/genfiles/view/server-host-
> name/regex_revalidate.config
>
> time_namelookup: 0.005
> time_connect: 0.062
> time_appconnect: 0.307
> time_pretransfer: 0.307
> time_redirect: 0.000
> time_starttransfer: 0.781
> ----------
> time_total: 0.781
>
> On Mon, Dec 12, 2016 at 9:03 PM, Steve Malenfant <sm...@gmail.com>
> wrote:
>
> > I'm OK with this. I think most of the revalidate functionality is not
> well
> > understood here... I guess I need to look into the API since we are using
> > the UI to issue revalidate today.
> >
> > On Mon, Dec 12, 2016 at 5:43 PM, Jeremy Mitchell <mitchell852@apache.org
> >
> > wrote:
> >
> > > I've created an issue to solicit some feedback regarding the removal of
> > > regex_revalidate.config from the file system. If you are not aware,
> > > regex_revalidate.config is the ATS config file used to "invalidate
> > > content". Storing regex_revalidate.config on the file system makes it
> > > harder to set up a highly-available, redundant traffic ops.
> > >
> > > https://issues.apache.org/jira/browse/TC-68
> > >
> > > Thanks!
> > >
> >
>
Re: regex_revalidate.config
Posted by Jeremy Mitchell <mi...@gmail.com>.
Obviously, it is going to be slower to pull regex_revalidate.config thru TO
as opposed to pulling it off the file system. The question is...is this
acceptable?
curling to
https://to.domain.com/Trafficserver-Snapshots/cdn-name/regex_revalidate.config
time_namelookup: 0.005
time_connect: 0.058
time_appconnect: 0.000
time_pretransfer: 0.000
time_redirect: 0.000
time_starttransfer: 0.000
----------
time_total: 0.063
curling to
https://to.domain.com/genfiles/view/server-host-name/regex_revalidate.config
time_namelookup: 0.005
time_connect: 0.062
time_appconnect: 0.307
time_pretransfer: 0.307
time_redirect: 0.000
time_starttransfer: 0.781
----------
time_total: 0.781
On Mon, Dec 12, 2016 at 9:03 PM, Steve Malenfant <sm...@gmail.com>
wrote:
> I'm OK with this. I think most of the revalidate functionality is not well
> understood here... I guess I need to look into the API since we are using
> the UI to issue revalidate today.
>
> On Mon, Dec 12, 2016 at 5:43 PM, Jeremy Mitchell <mi...@apache.org>
> wrote:
>
> > I've created an issue to solicit some feedback regarding the removal of
> > regex_revalidate.config from the file system. If you are not aware,
> > regex_revalidate.config is the ATS config file used to "invalidate
> > content". Storing regex_revalidate.config on the file system makes it
> > harder to set up a highly-available, redundant traffic ops.
> >
> > https://issues.apache.org/jira/browse/TC-68
> >
> > Thanks!
> >
>
Re: regex_revalidate.config
Posted by Steve Malenfant <sm...@gmail.com>.
I'm OK with this. I think most of the revalidate functionality is not well
understood here... I guess I need to look into the API since we are using
the UI to issue revalidate today.
On Mon, Dec 12, 2016 at 5:43 PM, Jeremy Mitchell <mi...@apache.org>
wrote:
> I've created an issue to solicit some feedback regarding the removal of
> regex_revalidate.config from the file system. If you are not aware,
> regex_revalidate.config is the ATS config file used to "invalidate
> content". Storing regex_revalidate.config on the file system makes it
> harder to set up a highly-available, redundant traffic ops.
>
> https://issues.apache.org/jira/browse/TC-68
>
> Thanks!
>