You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@trafficserver.apache.org by Evan Zelkowitz <ez...@apache.org> on 2020/07/21 21:24:33 UTC
Proposal - arbitrary remap reload based on config file API
There is an issue surrounding remap reloading when includes and plugin
config files are used. These will not trigger a remap reload on their
own. I think at this point everyone uses their external management
tools to issue a touch to the remap file to trigger a reload to
happen, but it's a bit of a work-around.
I'm proposing adding an API that will allow plugins to register their
config files to be monitored in the same way that remap and records
are currently monitored. When a traffic_ctl config reload is issued,
if one of these files has changed that can also trigger a remap
reload.
I have a POC going currently where I can add an arbitrary file to
trigger the reload but got to the point of looking at the API so
figured I would send out a list message for any ideas/concerns/etc
- Evan
Re: Proposal - arbitrary remap reload based on config file API
Posted by Evan Zelkowitz <ev...@gmail.com>.
Well another option is to go ahead with what I have, and add another
param like what we did for plugin reloading to allow this
functionality to be disabled. That way it is both dependent on the
plugin implementing the feature to begin with, so first this will not
affect custom plugins if you dont want it to, and then secondary to
that there is a blanket way to disable it
On Tue, Jul 21, 2020 at 8:30 PM Alan Carroll
<so...@verizonmedia.com> wrote:
>
> "Currently there is no way for a plugin to just reload its config file when it is running as a remap"
>
> Plugin Messages work for me - https://docs.trafficserver.apache.org/en/8.0.x/developer-guide/api/functions/TSLifecycleHookAdd.en.html#_CPPv3N17TSLifecycleHookID21TS_LIFECYCLE_MSG_HOOKE
>
> That's what TxnBox uses to signal a config reload. Also, as of ATS 9 and the new sets of remap plugin callbacks that happen at.the "remap.config" loading time, different instances of the same plugin in different rules can share state, and the timed probes could be based there (TxnBox uses this to share config files, so any particular config file is loaded and parsed into YAML only once - stole that from Zwoop).
>
> That said, it doesn't handle the case of changing the remap rule arguments, and does require config buffering logic inside the plugin which isn't always easy. In terms of reloading just a config for a plugin, this gets a bit intricate when the plugin is used in multiple remap rules each of which uses a subset of a number of configuration files. Does the core handle each of those, substituting it in *and* dealing with config buffering around live traffic? How would that work - would the core do a RemapNewInstance and RemapDestroyInstance? One reason TxnBox has just "reload" is I didn't want to deal with this complexity and possible cross config dependencies. I'm not enthused about trying to put that in the core and then debug it. If we're going to do anything, I'd go with Evan's plan and allow a plugin to add files that trigger a "remap.config" reload. It's not something that's required if it doesn't work in your deployment, it's simple to understand and implement. If you really need better, there are (as noted above) ways to do that.
>
> On Tue, Jul 21, 2020 at 6:33 PM Evan Zelkowitz <ez...@apache.org> wrote:
>>
>> Ok well in that case maybe this does make more sense to integrate as a
>> part of the plugindso code via its own APIs, such as the ones to
>> notify of a full remap reload pre/post. Instead have a new api in
>> there of a per-plugin config file that it can keep track of and notify
>> the plugin who registered it for updates
>>
>> Though the includes issue will have to be handled elsewhere
>>
>> On Tue, Jul 21, 2020 at 5:22 PM Sudheer Vinukonda
>> <su...@yahoo.com> wrote:
>> >
>> > I don't think periodic reload of config for regex_revalidate is optional. Pretty certain we used that functionality in the past (for purging tombstones that get added on the fly).
>> >
>> > Reloading just the config is not the same as reloading the remap DSO or conflicts with the remap reload even.
>> >
>> > The concern with having to reload the entire remap file is that, it is "unwarranted" and will result in reloading a giant remap file (for e.g our remap file is 20K+ lines long) for a change in some random plugin's config on a single remap line.
>> >
>> >
>> >
>> >
>> > On Tuesday, July 21, 2020, 04:09:14 PM PDT, Evan Zelkowitz <ez...@apache.org> wrote:
>> >
>> >
>> > regex reval first uses the tsmgmtupdate call in all cases, so it will
>> > be triggered by a config reload and since it's a global plugin it can
>> > do that, the periodic reload is optional. During the last summit
>> > there was a lot of concern over remap.config reloading on its own
>> > outside of config reload due to a bug that existed. Just having timed
>> > reloads would basically reintroduce that functionality just for that
>> > plugin for its configuration. Doing it this way also creates one
>> > single way to load a config, a plugin doesn't have to know if its
>> > reloading a config or monitoring anything, it is just destroyed and
>> > recreated and loads everything as it normally would in newinstance
>> >
>> > You also don't have to create duplicate reload code, or reinvent it,
>> > it would be a simple API call to register a file that should be
>> > monitored and that's it
>> >
>> > What is the concern with doing a remap reload in these cases though?
>> > Since it would essentially have to do the same thing if a plugin did
>> > not use a config file and had all of its parameters listed out as
>> > pparams to a plugin that would have changed a line in the remap
>> > config. So to me a plugins config file is equivalent to packaging up a
>> > bunch of pparams or having them represented in a different/logical
>> > way, but are still equivalent to a line of a bunch of pparams
>> >
>> > On Tue, Jul 21, 2020 at 4:54 PM Sudheer Vinukonda
>> > <su...@yahoo.com.invalid> wrote:
>> > >
>> > > Hmm..I am not sure to understand the concern with reloading individual config files outside of "config reload".
>> > > I think a reasonable approach is to use a background thread periodically polling for config file changes and load it in isolation (an example is regex_revalidate plugin).
>> > > https://github.com/apache/trafficserver/blob/master/plugins/regex_revalidate/regex_revalidate.c
>> > > What are the concerns with that approach?
>> > >
>> > >
>> > >
>> > > On Tuesday, July 21, 2020, 03:45:37 PM PDT, Evan Zelkowitz <ez...@apache.org> wrote:
>> > >
>> > > Currently there is no way for a plugin to just reload its config file
>> > > when it is running as a remap, unless you are running some sort of
>> > > timer or checking on every remap call. The mgmtupdate cannot be used
>> > > for remaps due to how it is designed. Using a timer you then start
>> > > reloading configs outside of the specified config reload call which
>> > > many do not like (since we had the previous issue of the remap
>> > > reloading on its own when it was changed). I suggest reloading the
>> > > remap because that keeps this within a well designed workflow that we
>> > > already know wrt to reference counting and deletion/instantiation of
>> > > plugins and what data they will be loaded with
>> > >
>> > > I suppose another option would be to add functionality to the
>> > > PluginDSO and have it do file monitoring as requested by plugins to
>> > > reload individual ones rather than just the blanket remap reload, but
>> > > currently anything dealing with config files like that is already
>> > > handled by the manager, this would be a large extension to the plugin
>> > > code
>> > >
>> > > On Tue, Jul 21, 2020 at 4:24 PM John Rushford <jj...@gmail.com> wrote:
>> > > >
>> > > > IMO a config file change should just cause the config to be reloaded no the entire remap or plugin DSO.
>> > > >
>> > > > Thanks
>> > > > John
>> > > >
>> > > > On Tue, Jul 21, 2020 at 4:03 PM Sudheer Vinukonda <su...@yahoo.com> wrote:
>> > > >>
>> > > >> I think it makes sense when "includes" are changed, but, I'm wondering if it makes sense to reload the remap file when a specific config file for a plugin is changed.
>> > > >>
>> > > >> Shouldn't a plugin config file change limit to reloading the plugin (or even just the config file for that plugin) as opposed to reloading the entire remap config?
>> > > >>
>> > > >> Thanks,
>> > > >>
>> > > >> Sudheer
>> > > >>
>> > > >>
>> > > >> On Tuesday, July 21, 2020, 02:24:53 PM PDT, Evan Zelkowitz <ez...@apache.org> wrote:
>> > > >>
>> > > >>
>> > > >> There is an issue surrounding remap reloading when includes and plugin
>> > > >> config files are used. These will not trigger a remap reload on their
>> > > >> own. I think at this point everyone uses their external management
>> > > >> tools to issue a touch to the remap file to trigger a reload to
>> > > >> happen, but it's a bit of a work-around.
>> > > >>
>> > > >> I'm proposing adding an API that will allow plugins to register their
>> > > >> config files to be monitored in the same way that remap and records
>> > > >> are currently monitored. When a traffic_ctl config reload is issued,
>> > > >> if one of these files has changed that can also trigger a remap
>> > > >> reload.
>> > > >>
>> > > >> I have a POC going currently where I can add an arbitrary file to
>> > > >> trigger the reload but got to the point of looking at the API so
>> > > >> figured I would send out a list message for any ideas/concerns/etc
>> > > >>
>> > > >> - Evan
>> > > >
>> > > >
>> > > >
>> > > > --
>> > > > John Rushford
>> > > > jjrushford@gmail.com
>> > >
Re: Proposal - arbitrary remap reload based on config file API
Posted by Alan Carroll <so...@verizonmedia.com>.
"Currently there is no way for a plugin to just reload its config file when
it is running as a remap"
Plugin Messages work for me -
https://docs.trafficserver.apache.org/en/8.0.x/developer-guide/api/functions/TSLifecycleHookAdd.en.html#_CPPv3N17TSLifecycleHookID21TS_LIFECYCLE_MSG_HOOKE
That's what TxnBox uses to signal a config reload. Also, as of ATS 9 and
the new sets of remap plugin callbacks that happen at.the "remap.config"
loading time, different instances of the same plugin in different rules can
share state, and the timed probes could be based there (TxnBox uses this to
share config files, so any particular config file is loaded and parsed into
YAML only once - stole that from Zwoop).
That said, it doesn't handle the case of changing the remap rule arguments,
and does require config buffering logic inside the plugin which isn't
always easy. In terms of reloading just a config for a plugin, this gets a
bit intricate when the plugin is used in multiple remap rules each of which
uses a subset of a number of configuration files. Does the core handle each
of those, substituting it in *and* dealing with config buffering around
live traffic? How would that work - would the core do a RemapNewInstance
and RemapDestroyInstance? One reason TxnBox has just "reload" is I didn't
want to deal with this complexity and possible cross config dependencies.
I'm not enthused about trying to put that in the core and then debug it. If
we're going to do anything, I'd go with Evan's plan and allow a plugin to
add files that trigger a "remap.config" reload. It's not something that's
required if it doesn't work in your deployment, it's simple to understand
and implement. If you really need better, there are (as noted above) ways
to do that.
On Tue, Jul 21, 2020 at 6:33 PM Evan Zelkowitz <ez...@apache.org> wrote:
> Ok well in that case maybe this does make more sense to integrate as a
> part of the plugindso code via its own APIs, such as the ones to
> notify of a full remap reload pre/post. Instead have a new api in
> there of a per-plugin config file that it can keep track of and notify
> the plugin who registered it for updates
>
> Though the includes issue will have to be handled elsewhere
>
> On Tue, Jul 21, 2020 at 5:22 PM Sudheer Vinukonda
> <su...@yahoo.com> wrote:
> >
> > I don't think periodic reload of config for regex_revalidate is
> optional. Pretty certain we used that functionality in the past (for
> purging tombstones that get added on the fly).
> >
> > Reloading just the config is not the same as reloading the remap DSO or
> conflicts with the remap reload even.
> >
> > The concern with having to reload the entire remap file is that, it is
> "unwarranted" and will result in reloading a giant remap file (for e.g our
> remap file is 20K+ lines long) for a change in some random plugin's config
> on a single remap line.
> >
> >
> >
> >
> > On Tuesday, July 21, 2020, 04:09:14 PM PDT, Evan Zelkowitz <
> eze@apache.org> wrote:
> >
> >
> > regex reval first uses the tsmgmtupdate call in all cases, so it will
> > be triggered by a config reload and since it's a global plugin it can
> > do that, the periodic reload is optional. During the last summit
> > there was a lot of concern over remap.config reloading on its own
> > outside of config reload due to a bug that existed. Just having timed
> > reloads would basically reintroduce that functionality just for that
> > plugin for its configuration. Doing it this way also creates one
> > single way to load a config, a plugin doesn't have to know if its
> > reloading a config or monitoring anything, it is just destroyed and
> > recreated and loads everything as it normally would in newinstance
> >
> > You also don't have to create duplicate reload code, or reinvent it,
> > it would be a simple API call to register a file that should be
> > monitored and that's it
> >
> > What is the concern with doing a remap reload in these cases though?
> > Since it would essentially have to do the same thing if a plugin did
> > not use a config file and had all of its parameters listed out as
> > pparams to a plugin that would have changed a line in the remap
> > config. So to me a plugins config file is equivalent to packaging up a
> > bunch of pparams or having them represented in a different/logical
> > way, but are still equivalent to a line of a bunch of pparams
> >
> > On Tue, Jul 21, 2020 at 4:54 PM Sudheer Vinukonda
> > <su...@yahoo.com.invalid> wrote:
> > >
> > > Hmm..I am not sure to understand the concern with reloading
> individual config files outside of "config reload".
> > > I think a reasonable approach is to use a background thread
> periodically polling for config file changes and load it in isolation (an
> example is regex_revalidate plugin).
> > >
> https://github.com/apache/trafficserver/blob/master/plugins/regex_revalidate/regex_revalidate.c
> > > What are the concerns with that approach?
> > >
> > >
> > >
> > > On Tuesday, July 21, 2020, 03:45:37 PM PDT, Evan Zelkowitz <
> eze@apache.org> wrote:
> > >
> > > Currently there is no way for a plugin to just reload its config file
> > > when it is running as a remap, unless you are running some sort of
> > > timer or checking on every remap call. The mgmtupdate cannot be used
> > > for remaps due to how it is designed. Using a timer you then start
> > > reloading configs outside of the specified config reload call which
> > > many do not like (since we had the previous issue of the remap
> > > reloading on its own when it was changed). I suggest reloading the
> > > remap because that keeps this within a well designed workflow that we
> > > already know wrt to reference counting and deletion/instantiation of
> > > plugins and what data they will be loaded with
> > >
> > > I suppose another option would be to add functionality to the
> > > PluginDSO and have it do file monitoring as requested by plugins to
> > > reload individual ones rather than just the blanket remap reload, but
> > > currently anything dealing with config files like that is already
> > > handled by the manager, this would be a large extension to the plugin
> > > code
> > >
> > > On Tue, Jul 21, 2020 at 4:24 PM John Rushford <jj...@gmail.com>
> wrote:
> > > >
> > > > IMO a config file change should just cause the config to be reloaded
> no the entire remap or plugin DSO.
> > > >
> > > > Thanks
> > > > John
> > > >
> > > > On Tue, Jul 21, 2020 at 4:03 PM Sudheer Vinukonda <
> sudheervinukonda@yahoo.com> wrote:
> > > >>
> > > >> I think it makes sense when "includes" are changed, but, I'm
> wondering if it makes sense to reload the remap file when a specific config
> file for a plugin is changed.
> > > >>
> > > >> Shouldn't a plugin config file change limit to reloading the plugin
> (or even just the config file for that plugin) as opposed to reloading the
> entire remap config?
> > > >>
> > > >> Thanks,
> > > >>
> > > >> Sudheer
> > > >>
> > > >>
> > > >> On Tuesday, July 21, 2020, 02:24:53 PM PDT, Evan Zelkowitz <
> eze@apache.org> wrote:
> > > >>
> > > >>
> > > >> There is an issue surrounding remap reloading when includes and
> plugin
> > > >> config files are used. These will not trigger a remap reload on
> their
> > > >> own. I think at this point everyone uses their external management
> > > >> tools to issue a touch to the remap file to trigger a reload to
> > > >> happen, but it's a bit of a work-around.
> > > >>
> > > >> I'm proposing adding an API that will allow plugins to register
> their
> > > >> config files to be monitored in the same way that remap and records
> > > >> are currently monitored. When a traffic_ctl config reload is
> issued,
> > > >> if one of these files has changed that can also trigger a remap
> > > >> reload.
> > > >>
> > > >> I have a POC going currently where I can add an arbitrary file to
> > > >> trigger the reload but got to the point of looking at the API so
> > > >> figured I would send out a list message for any ideas/concerns/etc
> > > >>
> > > >> - Evan
> > > >
> > > >
> > > >
> > > > --
> > > > John Rushford
> > > > jjrushford@gmail.com
> > >
>
Re: Proposal - arbitrary remap reload based on config file API
Posted by Evan Zelkowitz <ez...@apache.org>.
Ok well in that case maybe this does make more sense to integrate as a
part of the plugindso code via its own APIs, such as the ones to
notify of a full remap reload pre/post. Instead have a new api in
there of a per-plugin config file that it can keep track of and notify
the plugin who registered it for updates
Though the includes issue will have to be handled elsewhere
On Tue, Jul 21, 2020 at 5:22 PM Sudheer Vinukonda
<su...@yahoo.com> wrote:
>
> I don't think periodic reload of config for regex_revalidate is optional. Pretty certain we used that functionality in the past (for purging tombstones that get added on the fly).
>
> Reloading just the config is not the same as reloading the remap DSO or conflicts with the remap reload even.
>
> The concern with having to reload the entire remap file is that, it is "unwarranted" and will result in reloading a giant remap file (for e.g our remap file is 20K+ lines long) for a change in some random plugin's config on a single remap line.
>
>
>
>
> On Tuesday, July 21, 2020, 04:09:14 PM PDT, Evan Zelkowitz <ez...@apache.org> wrote:
>
>
> regex reval first uses the tsmgmtupdate call in all cases, so it will
> be triggered by a config reload and since it's a global plugin it can
> do that, the periodic reload is optional. During the last summit
> there was a lot of concern over remap.config reloading on its own
> outside of config reload due to a bug that existed. Just having timed
> reloads would basically reintroduce that functionality just for that
> plugin for its configuration. Doing it this way also creates one
> single way to load a config, a plugin doesn't have to know if its
> reloading a config or monitoring anything, it is just destroyed and
> recreated and loads everything as it normally would in newinstance
>
> You also don't have to create duplicate reload code, or reinvent it,
> it would be a simple API call to register a file that should be
> monitored and that's it
>
> What is the concern with doing a remap reload in these cases though?
> Since it would essentially have to do the same thing if a plugin did
> not use a config file and had all of its parameters listed out as
> pparams to a plugin that would have changed a line in the remap
> config. So to me a plugins config file is equivalent to packaging up a
> bunch of pparams or having them represented in a different/logical
> way, but are still equivalent to a line of a bunch of pparams
>
> On Tue, Jul 21, 2020 at 4:54 PM Sudheer Vinukonda
> <su...@yahoo.com.invalid> wrote:
> >
> > Hmm..I am not sure to understand the concern with reloading individual config files outside of "config reload".
> > I think a reasonable approach is to use a background thread periodically polling for config file changes and load it in isolation (an example is regex_revalidate plugin).
> > https://github.com/apache/trafficserver/blob/master/plugins/regex_revalidate/regex_revalidate.c
> > What are the concerns with that approach?
> >
> >
> >
> > On Tuesday, July 21, 2020, 03:45:37 PM PDT, Evan Zelkowitz <ez...@apache.org> wrote:
> >
> > Currently there is no way for a plugin to just reload its config file
> > when it is running as a remap, unless you are running some sort of
> > timer or checking on every remap call. The mgmtupdate cannot be used
> > for remaps due to how it is designed. Using a timer you then start
> > reloading configs outside of the specified config reload call which
> > many do not like (since we had the previous issue of the remap
> > reloading on its own when it was changed). I suggest reloading the
> > remap because that keeps this within a well designed workflow that we
> > already know wrt to reference counting and deletion/instantiation of
> > plugins and what data they will be loaded with
> >
> > I suppose another option would be to add functionality to the
> > PluginDSO and have it do file monitoring as requested by plugins to
> > reload individual ones rather than just the blanket remap reload, but
> > currently anything dealing with config files like that is already
> > handled by the manager, this would be a large extension to the plugin
> > code
> >
> > On Tue, Jul 21, 2020 at 4:24 PM John Rushford <jj...@gmail.com> wrote:
> > >
> > > IMO a config file change should just cause the config to be reloaded no the entire remap or plugin DSO.
> > >
> > > Thanks
> > > John
> > >
> > > On Tue, Jul 21, 2020 at 4:03 PM Sudheer Vinukonda <su...@yahoo.com> wrote:
> > >>
> > >> I think it makes sense when "includes" are changed, but, I'm wondering if it makes sense to reload the remap file when a specific config file for a plugin is changed.
> > >>
> > >> Shouldn't a plugin config file change limit to reloading the plugin (or even just the config file for that plugin) as opposed to reloading the entire remap config?
> > >>
> > >> Thanks,
> > >>
> > >> Sudheer
> > >>
> > >>
> > >> On Tuesday, July 21, 2020, 02:24:53 PM PDT, Evan Zelkowitz <ez...@apache.org> wrote:
> > >>
> > >>
> > >> There is an issue surrounding remap reloading when includes and plugin
> > >> config files are used. These will not trigger a remap reload on their
> > >> own. I think at this point everyone uses their external management
> > >> tools to issue a touch to the remap file to trigger a reload to
> > >> happen, but it's a bit of a work-around.
> > >>
> > >> I'm proposing adding an API that will allow plugins to register their
> > >> config files to be monitored in the same way that remap and records
> > >> are currently monitored. When a traffic_ctl config reload is issued,
> > >> if one of these files has changed that can also trigger a remap
> > >> reload.
> > >>
> > >> I have a POC going currently where I can add an arbitrary file to
> > >> trigger the reload but got to the point of looking at the API so
> > >> figured I would send out a list message for any ideas/concerns/etc
> > >>
> > >> - Evan
> > >
> > >
> > >
> > > --
> > > John Rushford
> > > jjrushford@gmail.com
> >
Re: Proposal - arbitrary remap reload based on config file API
Posted by Evan Zelkowitz <ez...@apache.org>.
Ok well in that case maybe this does make more sense to integrate as a
part of the plugindso code via its own APIs, such as the ones to
notify of a full remap reload pre/post. Instead have a new api in
there of a per-plugin config file that it can keep track of and notify
the plugin who registered it for updates
Though the includes issue will have to be handled elsewhere
On Tue, Jul 21, 2020 at 5:22 PM Sudheer Vinukonda
<su...@yahoo.com> wrote:
>
> I don't think periodic reload of config for regex_revalidate is optional. Pretty certain we used that functionality in the past (for purging tombstones that get added on the fly).
>
> Reloading just the config is not the same as reloading the remap DSO or conflicts with the remap reload even.
>
> The concern with having to reload the entire remap file is that, it is "unwarranted" and will result in reloading a giant remap file (for e.g our remap file is 20K+ lines long) for a change in some random plugin's config on a single remap line.
>
>
>
>
> On Tuesday, July 21, 2020, 04:09:14 PM PDT, Evan Zelkowitz <ez...@apache.org> wrote:
>
>
> regex reval first uses the tsmgmtupdate call in all cases, so it will
> be triggered by a config reload and since it's a global plugin it can
> do that, the periodic reload is optional. During the last summit
> there was a lot of concern over remap.config reloading on its own
> outside of config reload due to a bug that existed. Just having timed
> reloads would basically reintroduce that functionality just for that
> plugin for its configuration. Doing it this way also creates one
> single way to load a config, a plugin doesn't have to know if its
> reloading a config or monitoring anything, it is just destroyed and
> recreated and loads everything as it normally would in newinstance
>
> You also don't have to create duplicate reload code, or reinvent it,
> it would be a simple API call to register a file that should be
> monitored and that's it
>
> What is the concern with doing a remap reload in these cases though?
> Since it would essentially have to do the same thing if a plugin did
> not use a config file and had all of its parameters listed out as
> pparams to a plugin that would have changed a line in the remap
> config. So to me a plugins config file is equivalent to packaging up a
> bunch of pparams or having them represented in a different/logical
> way, but are still equivalent to a line of a bunch of pparams
>
> On Tue, Jul 21, 2020 at 4:54 PM Sudheer Vinukonda
> <su...@yahoo.com.invalid> wrote:
> >
> > Hmm..I am not sure to understand the concern with reloading individual config files outside of "config reload".
> > I think a reasonable approach is to use a background thread periodically polling for config file changes and load it in isolation (an example is regex_revalidate plugin).
> > https://github.com/apache/trafficserver/blob/master/plugins/regex_revalidate/regex_revalidate.c
> > What are the concerns with that approach?
> >
> >
> >
> > On Tuesday, July 21, 2020, 03:45:37 PM PDT, Evan Zelkowitz <ez...@apache.org> wrote:
> >
> > Currently there is no way for a plugin to just reload its config file
> > when it is running as a remap, unless you are running some sort of
> > timer or checking on every remap call. The mgmtupdate cannot be used
> > for remaps due to how it is designed. Using a timer you then start
> > reloading configs outside of the specified config reload call which
> > many do not like (since we had the previous issue of the remap
> > reloading on its own when it was changed). I suggest reloading the
> > remap because that keeps this within a well designed workflow that we
> > already know wrt to reference counting and deletion/instantiation of
> > plugins and what data they will be loaded with
> >
> > I suppose another option would be to add functionality to the
> > PluginDSO and have it do file monitoring as requested by plugins to
> > reload individual ones rather than just the blanket remap reload, but
> > currently anything dealing with config files like that is already
> > handled by the manager, this would be a large extension to the plugin
> > code
> >
> > On Tue, Jul 21, 2020 at 4:24 PM John Rushford <jj...@gmail.com> wrote:
> > >
> > > IMO a config file change should just cause the config to be reloaded no the entire remap or plugin DSO.
> > >
> > > Thanks
> > > John
> > >
> > > On Tue, Jul 21, 2020 at 4:03 PM Sudheer Vinukonda <su...@yahoo.com> wrote:
> > >>
> > >> I think it makes sense when "includes" are changed, but, I'm wondering if it makes sense to reload the remap file when a specific config file for a plugin is changed.
> > >>
> > >> Shouldn't a plugin config file change limit to reloading the plugin (or even just the config file for that plugin) as opposed to reloading the entire remap config?
> > >>
> > >> Thanks,
> > >>
> > >> Sudheer
> > >>
> > >>
> > >> On Tuesday, July 21, 2020, 02:24:53 PM PDT, Evan Zelkowitz <ez...@apache.org> wrote:
> > >>
> > >>
> > >> There is an issue surrounding remap reloading when includes and plugin
> > >> config files are used. These will not trigger a remap reload on their
> > >> own. I think at this point everyone uses their external management
> > >> tools to issue a touch to the remap file to trigger a reload to
> > >> happen, but it's a bit of a work-around.
> > >>
> > >> I'm proposing adding an API that will allow plugins to register their
> > >> config files to be monitored in the same way that remap and records
> > >> are currently monitored. When a traffic_ctl config reload is issued,
> > >> if one of these files has changed that can also trigger a remap
> > >> reload.
> > >>
> > >> I have a POC going currently where I can add an arbitrary file to
> > >> trigger the reload but got to the point of looking at the API so
> > >> figured I would send out a list message for any ideas/concerns/etc
> > >>
> > >> - Evan
> > >
> > >
> > >
> > > --
> > > John Rushford
> > > jjrushford@gmail.com
> >
Re: Proposal - arbitrary remap reload based on config file API
Posted by Sudheer Vinukonda <su...@yahoo.com>.
I don't think periodic reload of config for regex_revalidate is optional. Pretty certain we used that functionality in the past (for purging tombstones that get added on the fly).
Reloading just the config is not the same as reloading the remap DSO or conflicts with the remap reload even.
The concern with having to reload the entire remap file is that, it is "unwarranted" and will result in reloading a giant remap file (for e.g our remap file is 20K+ lines long) for a change in some random plugin's config on a single remap line.
On Tuesday, July 21, 2020, 04:09:14 PM PDT, Evan Zelkowitz <ez...@apache.org> wrote:
regex reval first uses the tsmgmtupdate call in all cases, so it will
be triggered by a config reload and since it's a global plugin it can
do that, the periodic reload is optional. During the last summit
there was a lot of concern over remap.config reloading on its own
outside of config reload due to a bug that existed. Just having timed
reloads would basically reintroduce that functionality just for that
plugin for its configuration. Doing it this way also creates one
single way to load a config, a plugin doesn't have to know if its
reloading a config or monitoring anything, it is just destroyed and
recreated and loads everything as it normally would in newinstance
You also don't have to create duplicate reload code, or reinvent it,
it would be a simple API call to register a file that should be
monitored and that's it
What is the concern with doing a remap reload in these cases though?
Since it would essentially have to do the same thing if a plugin did
not use a config file and had all of its parameters listed out as
pparams to a plugin that would have changed a line in the remap
config. So to me a plugins config file is equivalent to packaging up a
bunch of pparams or having them represented in a different/logical
way, but are still equivalent to a line of a bunch of pparams
On Tue, Jul 21, 2020 at 4:54 PM Sudheer Vinukonda
<su...@yahoo.com.invalid> wrote:
>
> Hmm..I am not sure to understand the concern with reloading individual config files outside of "config reload".
> I think a reasonable approach is to use a background thread periodically polling for config file changes and load it in isolation (an example is regex_revalidate plugin).
> https://github.com/apache/trafficserver/blob/master/plugins/regex_revalidate/regex_revalidate.c
> What are the concerns with that approach?
>
>
>
> On Tuesday, July 21, 2020, 03:45:37 PM PDT, Evan Zelkowitz <ez...@apache.org> wrote:
>
> Currently there is no way for a plugin to just reload its config file
> when it is running as a remap, unless you are running some sort of
> timer or checking on every remap call. The mgmtupdate cannot be used
> for remaps due to how it is designed. Using a timer you then start
> reloading configs outside of the specified config reload call which
> many do not like (since we had the previous issue of the remap
> reloading on its own when it was changed). I suggest reloading the
> remap because that keeps this within a well designed workflow that we
> already know wrt to reference counting and deletion/instantiation of
> plugins and what data they will be loaded with
>
> I suppose another option would be to add functionality to the
> PluginDSO and have it do file monitoring as requested by plugins to
> reload individual ones rather than just the blanket remap reload, but
> currently anything dealing with config files like that is already
> handled by the manager, this would be a large extension to the plugin
> code
>
> On Tue, Jul 21, 2020 at 4:24 PM John Rushford <jj...@gmail.com> wrote:
> >
> > IMO a config file change should just cause the config to be reloaded no the entire remap or plugin DSO.
> >
> > Thanks
> > John
> >
> > On Tue, Jul 21, 2020 at 4:03 PM Sudheer Vinukonda <su...@yahoo.com> wrote:
> >>
> >> I think it makes sense when "includes" are changed, but, I'm wondering if it makes sense to reload the remap file when a specific config file for a plugin is changed.
> >>
> >> Shouldn't a plugin config file change limit to reloading the plugin (or even just the config file for that plugin) as opposed to reloading the entire remap config?
> >>
> >> Thanks,
> >>
> >> Sudheer
> >>
> >>
> >> On Tuesday, July 21, 2020, 02:24:53 PM PDT, Evan Zelkowitz <ez...@apache.org> wrote:
> >>
> >>
> >> There is an issue surrounding remap reloading when includes and plugin
> >> config files are used. These will not trigger a remap reload on their
> >> own. I think at this point everyone uses their external management
> >> tools to issue a touch to the remap file to trigger a reload to
> >> happen, but it's a bit of a work-around.
> >>
> >> I'm proposing adding an API that will allow plugins to register their
> >> config files to be monitored in the same way that remap and records
> >> are currently monitored. When a traffic_ctl config reload is issued,
> >> if one of these files has changed that can also trigger a remap
> >> reload.
> >>
> >> I have a POC going currently where I can add an arbitrary file to
> >> trigger the reload but got to the point of looking at the API so
> >> figured I would send out a list message for any ideas/concerns/etc
> >>
> >> - Evan
> >
> >
> >
> > --
> > John Rushford
> > jjrushford@gmail.com
>
Re: Proposal - arbitrary remap reload based on config file API
Posted by Sudheer Vinukonda <su...@yahoo.com.INVALID>.
I don't think periodic reload of config for regex_revalidate is optional. Pretty certain we used that functionality in the past (for purging tombstones that get added on the fly).
Reloading just the config is not the same as reloading the remap DSO or conflicts with the remap reload even.
The concern with having to reload the entire remap file is that, it is "unwarranted" and will result in reloading a giant remap file (for e.g our remap file is 20K+ lines long) for a change in some random plugin's config on a single remap line.
On Tuesday, July 21, 2020, 04:09:14 PM PDT, Evan Zelkowitz <ez...@apache.org> wrote:
regex reval first uses the tsmgmtupdate call in all cases, so it will
be triggered by a config reload and since it's a global plugin it can
do that, the periodic reload is optional. During the last summit
there was a lot of concern over remap.config reloading on its own
outside of config reload due to a bug that existed. Just having timed
reloads would basically reintroduce that functionality just for that
plugin for its configuration. Doing it this way also creates one
single way to load a config, a plugin doesn't have to know if its
reloading a config or monitoring anything, it is just destroyed and
recreated and loads everything as it normally would in newinstance
You also don't have to create duplicate reload code, or reinvent it,
it would be a simple API call to register a file that should be
monitored and that's it
What is the concern with doing a remap reload in these cases though?
Since it would essentially have to do the same thing if a plugin did
not use a config file and had all of its parameters listed out as
pparams to a plugin that would have changed a line in the remap
config. So to me a plugins config file is equivalent to packaging up a
bunch of pparams or having them represented in a different/logical
way, but are still equivalent to a line of a bunch of pparams
On Tue, Jul 21, 2020 at 4:54 PM Sudheer Vinukonda
<su...@yahoo.com.invalid> wrote:
>
> Hmm..I am not sure to understand the concern with reloading individual config files outside of "config reload".
> I think a reasonable approach is to use a background thread periodically polling for config file changes and load it in isolation (an example is regex_revalidate plugin).
> https://github.com/apache/trafficserver/blob/master/plugins/regex_revalidate/regex_revalidate.c
> What are the concerns with that approach?
>
>
>
> On Tuesday, July 21, 2020, 03:45:37 PM PDT, Evan Zelkowitz <ez...@apache.org> wrote:
>
> Currently there is no way for a plugin to just reload its config file
> when it is running as a remap, unless you are running some sort of
> timer or checking on every remap call. The mgmtupdate cannot be used
> for remaps due to how it is designed. Using a timer you then start
> reloading configs outside of the specified config reload call which
> many do not like (since we had the previous issue of the remap
> reloading on its own when it was changed). I suggest reloading the
> remap because that keeps this within a well designed workflow that we
> already know wrt to reference counting and deletion/instantiation of
> plugins and what data they will be loaded with
>
> I suppose another option would be to add functionality to the
> PluginDSO and have it do file monitoring as requested by plugins to
> reload individual ones rather than just the blanket remap reload, but
> currently anything dealing with config files like that is already
> handled by the manager, this would be a large extension to the plugin
> code
>
> On Tue, Jul 21, 2020 at 4:24 PM John Rushford <jj...@gmail.com> wrote:
> >
> > IMO a config file change should just cause the config to be reloaded no the entire remap or plugin DSO.
> >
> > Thanks
> > John
> >
> > On Tue, Jul 21, 2020 at 4:03 PM Sudheer Vinukonda <su...@yahoo.com> wrote:
> >>
> >> I think it makes sense when "includes" are changed, but, I'm wondering if it makes sense to reload the remap file when a specific config file for a plugin is changed.
> >>
> >> Shouldn't a plugin config file change limit to reloading the plugin (or even just the config file for that plugin) as opposed to reloading the entire remap config?
> >>
> >> Thanks,
> >>
> >> Sudheer
> >>
> >>
> >> On Tuesday, July 21, 2020, 02:24:53 PM PDT, Evan Zelkowitz <ez...@apache.org> wrote:
> >>
> >>
> >> There is an issue surrounding remap reloading when includes and plugin
> >> config files are used. These will not trigger a remap reload on their
> >> own. I think at this point everyone uses their external management
> >> tools to issue a touch to the remap file to trigger a reload to
> >> happen, but it's a bit of a work-around.
> >>
> >> I'm proposing adding an API that will allow plugins to register their
> >> config files to be monitored in the same way that remap and records
> >> are currently monitored. When a traffic_ctl config reload is issued,
> >> if one of these files has changed that can also trigger a remap
> >> reload.
> >>
> >> I have a POC going currently where I can add an arbitrary file to
> >> trigger the reload but got to the point of looking at the API so
> >> figured I would send out a list message for any ideas/concerns/etc
> >>
> >> - Evan
> >
> >
> >
> > --
> > John Rushford
> > jjrushford@gmail.com
>
Re: Proposal - arbitrary remap reload based on config file API
Posted by Evan Zelkowitz <ez...@apache.org>.
regex reval first uses the tsmgmtupdate call in all cases, so it will
be triggered by a config reload and since it's a global plugin it can
do that, the periodic reload is optional. During the last summit
there was a lot of concern over remap.config reloading on its own
outside of config reload due to a bug that existed. Just having timed
reloads would basically reintroduce that functionality just for that
plugin for its configuration. Doing it this way also creates one
single way to load a config, a plugin doesn't have to know if its
reloading a config or monitoring anything, it is just destroyed and
recreated and loads everything as it normally would in newinstance
You also don't have to create duplicate reload code, or reinvent it,
it would be a simple API call to register a file that should be
monitored and that's it
What is the concern with doing a remap reload in these cases though?
Since it would essentially have to do the same thing if a plugin did
not use a config file and had all of its parameters listed out as
pparams to a plugin that would have changed a line in the remap
config. So to me a plugins config file is equivalent to packaging up a
bunch of pparams or having them represented in a different/logical
way, but are still equivalent to a line of a bunch of pparams
On Tue, Jul 21, 2020 at 4:54 PM Sudheer Vinukonda
<su...@yahoo.com.invalid> wrote:
>
> Hmm..I am not sure to understand the concern with reloading individual config files outside of "config reload".
> I think a reasonable approach is to use a background thread periodically polling for config file changes and load it in isolation (an example is regex_revalidate plugin).
> https://github.com/apache/trafficserver/blob/master/plugins/regex_revalidate/regex_revalidate.c
> What are the concerns with that approach?
>
>
>
> On Tuesday, July 21, 2020, 03:45:37 PM PDT, Evan Zelkowitz <ez...@apache.org> wrote:
>
> Currently there is no way for a plugin to just reload its config file
> when it is running as a remap, unless you are running some sort of
> timer or checking on every remap call. The mgmtupdate cannot be used
> for remaps due to how it is designed. Using a timer you then start
> reloading configs outside of the specified config reload call which
> many do not like (since we had the previous issue of the remap
> reloading on its own when it was changed). I suggest reloading the
> remap because that keeps this within a well designed workflow that we
> already know wrt to reference counting and deletion/instantiation of
> plugins and what data they will be loaded with
>
> I suppose another option would be to add functionality to the
> PluginDSO and have it do file monitoring as requested by plugins to
> reload individual ones rather than just the blanket remap reload, but
> currently anything dealing with config files like that is already
> handled by the manager, this would be a large extension to the plugin
> code
>
> On Tue, Jul 21, 2020 at 4:24 PM John Rushford <jj...@gmail.com> wrote:
> >
> > IMO a config file change should just cause the config to be reloaded no the entire remap or plugin DSO.
> >
> > Thanks
> > John
> >
> > On Tue, Jul 21, 2020 at 4:03 PM Sudheer Vinukonda <su...@yahoo.com> wrote:
> >>
> >> I think it makes sense when "includes" are changed, but, I'm wondering if it makes sense to reload the remap file when a specific config file for a plugin is changed.
> >>
> >> Shouldn't a plugin config file change limit to reloading the plugin (or even just the config file for that plugin) as opposed to reloading the entire remap config?
> >>
> >> Thanks,
> >>
> >> Sudheer
> >>
> >>
> >> On Tuesday, July 21, 2020, 02:24:53 PM PDT, Evan Zelkowitz <ez...@apache.org> wrote:
> >>
> >>
> >> There is an issue surrounding remap reloading when includes and plugin
> >> config files are used. These will not trigger a remap reload on their
> >> own. I think at this point everyone uses their external management
> >> tools to issue a touch to the remap file to trigger a reload to
> >> happen, but it's a bit of a work-around.
> >>
> >> I'm proposing adding an API that will allow plugins to register their
> >> config files to be monitored in the same way that remap and records
> >> are currently monitored. When a traffic_ctl config reload is issued,
> >> if one of these files has changed that can also trigger a remap
> >> reload.
> >>
> >> I have a POC going currently where I can add an arbitrary file to
> >> trigger the reload but got to the point of looking at the API so
> >> figured I would send out a list message for any ideas/concerns/etc
> >>
> >> - Evan
> >
> >
> >
> > --
> > John Rushford
> > jjrushford@gmail.com
>
Re: Proposal - arbitrary remap reload based on config file API
Posted by Evan Zelkowitz <ez...@apache.org>.
regex reval first uses the tsmgmtupdate call in all cases, so it will
be triggered by a config reload and since it's a global plugin it can
do that, the periodic reload is optional. During the last summit
there was a lot of concern over remap.config reloading on its own
outside of config reload due to a bug that existed. Just having timed
reloads would basically reintroduce that functionality just for that
plugin for its configuration. Doing it this way also creates one
single way to load a config, a plugin doesn't have to know if its
reloading a config or monitoring anything, it is just destroyed and
recreated and loads everything as it normally would in newinstance
You also don't have to create duplicate reload code, or reinvent it,
it would be a simple API call to register a file that should be
monitored and that's it
What is the concern with doing a remap reload in these cases though?
Since it would essentially have to do the same thing if a plugin did
not use a config file and had all of its parameters listed out as
pparams to a plugin that would have changed a line in the remap
config. So to me a plugins config file is equivalent to packaging up a
bunch of pparams or having them represented in a different/logical
way, but are still equivalent to a line of a bunch of pparams
On Tue, Jul 21, 2020 at 4:54 PM Sudheer Vinukonda
<su...@yahoo.com.invalid> wrote:
>
> Hmm..I am not sure to understand the concern with reloading individual config files outside of "config reload".
> I think a reasonable approach is to use a background thread periodically polling for config file changes and load it in isolation (an example is regex_revalidate plugin).
> https://github.com/apache/trafficserver/blob/master/plugins/regex_revalidate/regex_revalidate.c
> What are the concerns with that approach?
>
>
>
> On Tuesday, July 21, 2020, 03:45:37 PM PDT, Evan Zelkowitz <ez...@apache.org> wrote:
>
> Currently there is no way for a plugin to just reload its config file
> when it is running as a remap, unless you are running some sort of
> timer or checking on every remap call. The mgmtupdate cannot be used
> for remaps due to how it is designed. Using a timer you then start
> reloading configs outside of the specified config reload call which
> many do not like (since we had the previous issue of the remap
> reloading on its own when it was changed). I suggest reloading the
> remap because that keeps this within a well designed workflow that we
> already know wrt to reference counting and deletion/instantiation of
> plugins and what data they will be loaded with
>
> I suppose another option would be to add functionality to the
> PluginDSO and have it do file monitoring as requested by plugins to
> reload individual ones rather than just the blanket remap reload, but
> currently anything dealing with config files like that is already
> handled by the manager, this would be a large extension to the plugin
> code
>
> On Tue, Jul 21, 2020 at 4:24 PM John Rushford <jj...@gmail.com> wrote:
> >
> > IMO a config file change should just cause the config to be reloaded no the entire remap or plugin DSO.
> >
> > Thanks
> > John
> >
> > On Tue, Jul 21, 2020 at 4:03 PM Sudheer Vinukonda <su...@yahoo.com> wrote:
> >>
> >> I think it makes sense when "includes" are changed, but, I'm wondering if it makes sense to reload the remap file when a specific config file for a plugin is changed.
> >>
> >> Shouldn't a plugin config file change limit to reloading the plugin (or even just the config file for that plugin) as opposed to reloading the entire remap config?
> >>
> >> Thanks,
> >>
> >> Sudheer
> >>
> >>
> >> On Tuesday, July 21, 2020, 02:24:53 PM PDT, Evan Zelkowitz <ez...@apache.org> wrote:
> >>
> >>
> >> There is an issue surrounding remap reloading when includes and plugin
> >> config files are used. These will not trigger a remap reload on their
> >> own. I think at this point everyone uses their external management
> >> tools to issue a touch to the remap file to trigger a reload to
> >> happen, but it's a bit of a work-around.
> >>
> >> I'm proposing adding an API that will allow plugins to register their
> >> config files to be monitored in the same way that remap and records
> >> are currently monitored. When a traffic_ctl config reload is issued,
> >> if one of these files has changed that can also trigger a remap
> >> reload.
> >>
> >> I have a POC going currently where I can add an arbitrary file to
> >> trigger the reload but got to the point of looking at the API so
> >> figured I would send out a list message for any ideas/concerns/etc
> >>
> >> - Evan
> >
> >
> >
> > --
> > John Rushford
> > jjrushford@gmail.com
>
Re: Proposal - arbitrary remap reload based on config file API
Posted by Sudheer Vinukonda <su...@yahoo.com.INVALID>.
Hmm..I am not sure to understand the concern with reloading individual config files outside of "config reload".
I think a reasonable approach is to use a background thread periodically polling for config file changes and load it in isolation (an example is regex_revalidate plugin).
https://github.com/apache/trafficserver/blob/master/plugins/regex_revalidate/regex_revalidate.c
What are the concerns with that approach?
On Tuesday, July 21, 2020, 03:45:37 PM PDT, Evan Zelkowitz <ez...@apache.org> wrote:
Currently there is no way for a plugin to just reload its config file
when it is running as a remap, unless you are running some sort of
timer or checking on every remap call. The mgmtupdate cannot be used
for remaps due to how it is designed. Using a timer you then start
reloading configs outside of the specified config reload call which
many do not like (since we had the previous issue of the remap
reloading on its own when it was changed). I suggest reloading the
remap because that keeps this within a well designed workflow that we
already know wrt to reference counting and deletion/instantiation of
plugins and what data they will be loaded with
I suppose another option would be to add functionality to the
PluginDSO and have it do file monitoring as requested by plugins to
reload individual ones rather than just the blanket remap reload, but
currently anything dealing with config files like that is already
handled by the manager, this would be a large extension to the plugin
code
On Tue, Jul 21, 2020 at 4:24 PM John Rushford <jj...@gmail.com> wrote:
>
> IMO a config file change should just cause the config to be reloaded no the entire remap or plugin DSO.
>
> Thanks
> John
>
> On Tue, Jul 21, 2020 at 4:03 PM Sudheer Vinukonda <su...@yahoo.com> wrote:
>>
>> I think it makes sense when "includes" are changed, but, I'm wondering if it makes sense to reload the remap file when a specific config file for a plugin is changed.
>>
>> Shouldn't a plugin config file change limit to reloading the plugin (or even just the config file for that plugin) as opposed to reloading the entire remap config?
>>
>> Thanks,
>>
>> Sudheer
>>
>>
>> On Tuesday, July 21, 2020, 02:24:53 PM PDT, Evan Zelkowitz <ez...@apache.org> wrote:
>>
>>
>> There is an issue surrounding remap reloading when includes and plugin
>> config files are used. These will not trigger a remap reload on their
>> own. I think at this point everyone uses their external management
>> tools to issue a touch to the remap file to trigger a reload to
>> happen, but it's a bit of a work-around.
>>
>> I'm proposing adding an API that will allow plugins to register their
>> config files to be monitored in the same way that remap and records
>> are currently monitored. When a traffic_ctl config reload is issued,
>> if one of these files has changed that can also trigger a remap
>> reload.
>>
>> I have a POC going currently where I can add an arbitrary file to
>> trigger the reload but got to the point of looking at the API so
>> figured I would send out a list message for any ideas/concerns/etc
>>
>> - Evan
>
>
>
> --
> John Rushford
> jjrushford@gmail.com
Re: Proposal - arbitrary remap reload based on config file API
Posted by Sudheer Vinukonda <su...@yahoo.com>.
Hmm..I am not sure to understand the concern with reloading individual config files outside of "config reload".
I think a reasonable approach is to use a background thread periodically polling for config file changes and load it in isolation (an example is regex_revalidate plugin).
https://github.com/apache/trafficserver/blob/master/plugins/regex_revalidate/regex_revalidate.c
What are the concerns with that approach?
On Tuesday, July 21, 2020, 03:45:37 PM PDT, Evan Zelkowitz <ez...@apache.org> wrote:
Currently there is no way for a plugin to just reload its config file
when it is running as a remap, unless you are running some sort of
timer or checking on every remap call. The mgmtupdate cannot be used
for remaps due to how it is designed. Using a timer you then start
reloading configs outside of the specified config reload call which
many do not like (since we had the previous issue of the remap
reloading on its own when it was changed). I suggest reloading the
remap because that keeps this within a well designed workflow that we
already know wrt to reference counting and deletion/instantiation of
plugins and what data they will be loaded with
I suppose another option would be to add functionality to the
PluginDSO and have it do file monitoring as requested by plugins to
reload individual ones rather than just the blanket remap reload, but
currently anything dealing with config files like that is already
handled by the manager, this would be a large extension to the plugin
code
On Tue, Jul 21, 2020 at 4:24 PM John Rushford <jj...@gmail.com> wrote:
>
> IMO a config file change should just cause the config to be reloaded no the entire remap or plugin DSO.
>
> Thanks
> John
>
> On Tue, Jul 21, 2020 at 4:03 PM Sudheer Vinukonda <su...@yahoo.com> wrote:
>>
>> I think it makes sense when "includes" are changed, but, I'm wondering if it makes sense to reload the remap file when a specific config file for a plugin is changed.
>>
>> Shouldn't a plugin config file change limit to reloading the plugin (or even just the config file for that plugin) as opposed to reloading the entire remap config?
>>
>> Thanks,
>>
>> Sudheer
>>
>>
>> On Tuesday, July 21, 2020, 02:24:53 PM PDT, Evan Zelkowitz <ez...@apache.org> wrote:
>>
>>
>> There is an issue surrounding remap reloading when includes and plugin
>> config files are used. These will not trigger a remap reload on their
>> own. I think at this point everyone uses their external management
>> tools to issue a touch to the remap file to trigger a reload to
>> happen, but it's a bit of a work-around.
>>
>> I'm proposing adding an API that will allow plugins to register their
>> config files to be monitored in the same way that remap and records
>> are currently monitored. When a traffic_ctl config reload is issued,
>> if one of these files has changed that can also trigger a remap
>> reload.
>>
>> I have a POC going currently where I can add an arbitrary file to
>> trigger the reload but got to the point of looking at the API so
>> figured I would send out a list message for any ideas/concerns/etc
>>
>> - Evan
>
>
>
> --
> John Rushford
> jjrushford@gmail.com
Re: Proposal - arbitrary remap reload based on config file API
Posted by Evan Zelkowitz <ez...@apache.org>.
Currently there is no way for a plugin to just reload its config file
when it is running as a remap, unless you are running some sort of
timer or checking on every remap call. The mgmtupdate cannot be used
for remaps due to how it is designed. Using a timer you then start
reloading configs outside of the specified config reload call which
many do not like (since we had the previous issue of the remap
reloading on its own when it was changed). I suggest reloading the
remap because that keeps this within a well designed workflow that we
already know wrt to reference counting and deletion/instantiation of
plugins and what data they will be loaded with
I suppose another option would be to add functionality to the
PluginDSO and have it do file monitoring as requested by plugins to
reload individual ones rather than just the blanket remap reload, but
currently anything dealing with config files like that is already
handled by the manager, this would be a large extension to the plugin
code
On Tue, Jul 21, 2020 at 4:24 PM John Rushford <jj...@gmail.com> wrote:
>
> IMO a config file change should just cause the config to be reloaded no the entire remap or plugin DSO.
>
> Thanks
> John
>
> On Tue, Jul 21, 2020 at 4:03 PM Sudheer Vinukonda <su...@yahoo.com> wrote:
>>
>> I think it makes sense when "includes" are changed, but, I'm wondering if it makes sense to reload the remap file when a specific config file for a plugin is changed.
>>
>> Shouldn't a plugin config file change limit to reloading the plugin (or even just the config file for that plugin) as opposed to reloading the entire remap config?
>>
>> Thanks,
>>
>> Sudheer
>>
>>
>> On Tuesday, July 21, 2020, 02:24:53 PM PDT, Evan Zelkowitz <ez...@apache.org> wrote:
>>
>>
>> There is an issue surrounding remap reloading when includes and plugin
>> config files are used. These will not trigger a remap reload on their
>> own. I think at this point everyone uses their external management
>> tools to issue a touch to the remap file to trigger a reload to
>> happen, but it's a bit of a work-around.
>>
>> I'm proposing adding an API that will allow plugins to register their
>> config files to be monitored in the same way that remap and records
>> are currently monitored. When a traffic_ctl config reload is issued,
>> if one of these files has changed that can also trigger a remap
>> reload.
>>
>> I have a POC going currently where I can add an arbitrary file to
>> trigger the reload but got to the point of looking at the API so
>> figured I would send out a list message for any ideas/concerns/etc
>>
>> - Evan
>
>
>
> --
> John Rushford
> jjrushford@gmail.com
Re: Proposal - arbitrary remap reload based on config file API
Posted by Evan Zelkowitz <ez...@apache.org>.
Currently there is no way for a plugin to just reload its config file
when it is running as a remap, unless you are running some sort of
timer or checking on every remap call. The mgmtupdate cannot be used
for remaps due to how it is designed. Using a timer you then start
reloading configs outside of the specified config reload call which
many do not like (since we had the previous issue of the remap
reloading on its own when it was changed). I suggest reloading the
remap because that keeps this within a well designed workflow that we
already know wrt to reference counting and deletion/instantiation of
plugins and what data they will be loaded with
I suppose another option would be to add functionality to the
PluginDSO and have it do file monitoring as requested by plugins to
reload individual ones rather than just the blanket remap reload, but
currently anything dealing with config files like that is already
handled by the manager, this would be a large extension to the plugin
code
On Tue, Jul 21, 2020 at 4:24 PM John Rushford <jj...@gmail.com> wrote:
>
> IMO a config file change should just cause the config to be reloaded no the entire remap or plugin DSO.
>
> Thanks
> John
>
> On Tue, Jul 21, 2020 at 4:03 PM Sudheer Vinukonda <su...@yahoo.com> wrote:
>>
>> I think it makes sense when "includes" are changed, but, I'm wondering if it makes sense to reload the remap file when a specific config file for a plugin is changed.
>>
>> Shouldn't a plugin config file change limit to reloading the plugin (or even just the config file for that plugin) as opposed to reloading the entire remap config?
>>
>> Thanks,
>>
>> Sudheer
>>
>>
>> On Tuesday, July 21, 2020, 02:24:53 PM PDT, Evan Zelkowitz <ez...@apache.org> wrote:
>>
>>
>> There is an issue surrounding remap reloading when includes and plugin
>> config files are used. These will not trigger a remap reload on their
>> own. I think at this point everyone uses their external management
>> tools to issue a touch to the remap file to trigger a reload to
>> happen, but it's a bit of a work-around.
>>
>> I'm proposing adding an API that will allow plugins to register their
>> config files to be monitored in the same way that remap and records
>> are currently monitored. When a traffic_ctl config reload is issued,
>> if one of these files has changed that can also trigger a remap
>> reload.
>>
>> I have a POC going currently where I can add an arbitrary file to
>> trigger the reload but got to the point of looking at the API so
>> figured I would send out a list message for any ideas/concerns/etc
>>
>> - Evan
>
>
>
> --
> John Rushford
> jjrushford@gmail.com
Re: Proposal - arbitrary remap reload based on config file API
Posted by John Rushford <jj...@gmail.com>.
IMO a config file change should just cause the config to be reloaded no the
entire remap or plugin DSO.
Thanks
John
On Tue, Jul 21, 2020 at 4:03 PM Sudheer Vinukonda <
sudheervinukonda@yahoo.com> wrote:
> I think it makes sense when "includes" are changed, but, I'm wondering if
> it makes sense to reload the remap file when a specific config file for a
> plugin is changed.
>
> Shouldn't a plugin config file change limit to reloading the plugin (or
> even just the config file for that plugin) as opposed to reloading the
> entire remap config?
>
> Thanks,
>
> Sudheer
>
>
> On Tuesday, July 21, 2020, 02:24:53 PM PDT, Evan Zelkowitz <ez...@apache.org>
> wrote:
>
>
> There is an issue surrounding remap reloading when includes and plugin
> config files are used. These will not trigger a remap reload on their
> own. I think at this point everyone uses their external management
> tools to issue a touch to the remap file to trigger a reload to
> happen, but it's a bit of a work-around.
>
> I'm proposing adding an API that will allow plugins to register their
> config files to be monitored in the same way that remap and records
> are currently monitored. When a traffic_ctl config reload is issued,
> if one of these files has changed that can also trigger a remap
> reload.
>
> I have a POC going currently where I can add an arbitrary file to
> trigger the reload but got to the point of looking at the API so
> figured I would send out a list message for any ideas/concerns/etc
>
> - Evan
>
--
John Rushford
jjrushford@gmail.com
Re: Proposal - arbitrary remap reload based on config file API
Posted by John Rushford <jj...@gmail.com>.
IMO a config file change should just cause the config to be reloaded no the
entire remap or plugin DSO.
Thanks
John
On Tue, Jul 21, 2020 at 4:03 PM Sudheer Vinukonda <
sudheervinukonda@yahoo.com> wrote:
> I think it makes sense when "includes" are changed, but, I'm wondering if
> it makes sense to reload the remap file when a specific config file for a
> plugin is changed.
>
> Shouldn't a plugin config file change limit to reloading the plugin (or
> even just the config file for that plugin) as opposed to reloading the
> entire remap config?
>
> Thanks,
>
> Sudheer
>
>
> On Tuesday, July 21, 2020, 02:24:53 PM PDT, Evan Zelkowitz <ez...@apache.org>
> wrote:
>
>
> There is an issue surrounding remap reloading when includes and plugin
> config files are used. These will not trigger a remap reload on their
> own. I think at this point everyone uses their external management
> tools to issue a touch to the remap file to trigger a reload to
> happen, but it's a bit of a work-around.
>
> I'm proposing adding an API that will allow plugins to register their
> config files to be monitored in the same way that remap and records
> are currently monitored. When a traffic_ctl config reload is issued,
> if one of these files has changed that can also trigger a remap
> reload.
>
> I have a POC going currently where I can add an arbitrary file to
> trigger the reload but got to the point of looking at the API so
> figured I would send out a list message for any ideas/concerns/etc
>
> - Evan
>
--
John Rushford
jjrushford@gmail.com
Re: Proposal - arbitrary remap reload based on config file API
Posted by Sudheer Vinukonda <su...@yahoo.com.INVALID>.
I think it makes sense when "includes" are changed, but, I'm wondering if it makes sense to reload the remap file when a specific config file for a plugin is changed.
Shouldn't a plugin config file change limit to reloading the plugin (or even just the config file for that plugin) as opposed to reloading the entire remap config?
Thanks,
Sudheer
On Tuesday, July 21, 2020, 02:24:53 PM PDT, Evan Zelkowitz <ez...@apache.org> wrote:
There is an issue surrounding remap reloading when includes and plugin
config files are used. These will not trigger a remap reload on their
own. I think at this point everyone uses their external management
tools to issue a touch to the remap file to trigger a reload to
happen, but it's a bit of a work-around.
I'm proposing adding an API that will allow plugins to register their
config files to be monitored in the same way that remap and records
are currently monitored. When a traffic_ctl config reload is issued,
if one of these files has changed that can also trigger a remap
reload.
I have a POC going currently where I can add an arbitrary file to
trigger the reload but got to the point of looking at the API so
figured I would send out a list message for any ideas/concerns/etc
- Evan
Re: Proposal - arbitrary remap reload based on config file API
Posted by Sudheer Vinukonda <su...@yahoo.com>.
I think it makes sense when "includes" are changed, but, I'm wondering if it makes sense to reload the remap file when a specific config file for a plugin is changed.
Shouldn't a plugin config file change limit to reloading the plugin (or even just the config file for that plugin) as opposed to reloading the entire remap config?
Thanks,
Sudheer
On Tuesday, July 21, 2020, 02:24:53 PM PDT, Evan Zelkowitz <ez...@apache.org> wrote:
There is an issue surrounding remap reloading when includes and plugin
config files are used. These will not trigger a remap reload on their
own. I think at this point everyone uses their external management
tools to issue a touch to the remap file to trigger a reload to
happen, but it's a bit of a work-around.
I'm proposing adding an API that will allow plugins to register their
config files to be monitored in the same way that remap and records
are currently monitored. When a traffic_ctl config reload is issued,
if one of these files has changed that can also trigger a remap
reload.
I have a POC going currently where I can add an arbitrary file to
trigger the reload but got to the point of looking at the API so
figured I would send out a list message for any ideas/concerns/etc
- Evan