You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@trafficserver.apache.org by Alan Carroll <so...@yahoo-inc.com.INVALID> on 2016/11/21 15:20:37 UTC

Config variable source values

The config variable sourcing data was originally put in to be able to detect that a variable was set even if that setting was the same as the default (otherwise this is impossible to reliably determine). I am working with a plugin that defines its own config records but as far as I can tell (1) the source value for a plugin configured record default is the same as for an explicitly set value and (2) the source information is not available via the plugin API.
I would like to fix both of these.
First, add a "PLUGIN" value to RecSourceT to indicate the value is set as a default by the plugin and use EXPLICIT only for values from configuration files or explicit value setting API calls.
Second,
* Move or copy the RecSourceT define to apidef.h.in.* Add TSMgmtSourceGet(const char*) to return the RecSourceT value for a configuration variable.

Re: Config variable source values

Posted by James Peach <jp...@apache.org>.

> On Nov 21, 2016, at 2:04 PM, Alan Carroll <so...@yahoo-inc.com.INVALID> wrote:
> 
> But this is precisely the scenario in mind when the original RecSourceT code was put in for core. Otherwise, it's not possible to support the use of an old configuration with the new code. The default values of B and C will take precedence over value set for A in records.config even if neither B nor C is mentioned in the actual configuration of ATS. Even you write "Fallback to A would make sense if B and C are not present" but without this change, you can't do that.

You can; you use a designated missing value (NULL, -1, etc) as the default. Alternatively, have an explicit foo.is_enabled value (defaulting to disabled), which we do all the time.


I think that it is *really* confusing to distinguish between “foo=1” as the default and “foo=1” as “I actually mean it. I can’t even imagine how we would document the behaviour of plugins or configuration settings that take advantage of this.

> 
>    On Monday, November 21, 2016 3:37 PM, James Peach <jp...@apache.org> wrote:
> 
> 
> 
>> On Nov 21, 2016, at 1:22 PM, Alan Carroll <so...@yahoo-inc.com.INVALID> wrote:
>> 
>> The use case is providing backwards compatibility. Suppose a plugin uses config value A. A newer version, in order to support additional features, uses values B and C. It would be nice to be able to detect that neither B nor C were explicitly set by the administrator in records.config and therefore the plugin should fall back and use A. It is also required to *not* fall back to A if the administrator explicitly set B or C to the default value. Therefore checking the retrieved value against the default value is insufficient. One might use a bogus default value then, but now there's a problem if none of A,B, or C was set.
> 
> I don’t think this is sane :)
> 
> There’s no effective difference between a default value and a configured value. Mentioning a configuration value in `records.config` doesn’t mean “really make this so”, it just sets that value. So the newer plugin should just use B and C. Fallback to A would make sense if B and C are not present (eg. the value is a NULL string or some designared int), but this is specific to the plugin and unrelated to where the configuration value was set. Ad default is just as legitimate as an explicit configuration.
> 
> "It is also required to *not* fall back to A if the administrator explicitly set B or C to the default value”
> 
> This requirement leads to extremely confusing behavior and IMHO it is best to not go there.

Re: Config variable source values

Posted by Alan Carroll <so...@yahoo-inc.com.INVALID>.

But this is precisely the scenario in mind when the original RecSourceT code was put in for core. Otherwise, it's not possible to support the use of an old configuration with the new code. The default values of B and C will take precedence over value set for A in records.config even if neither B nor C is mentioned in the actual configuration of ATS. Even you write "Fallback to A would make sense if B and C are not present" but without this change, you can't do that.

    On Monday, November 21, 2016 3:37 PM, James Peach <jp...@apache.org> wrote:

> On Nov 21, 2016, at 1:22 PM, Alan Carroll <so...@yahoo-inc.com.INVALID> wrote:
> 
> The use case is providing backwards compatibility. Suppose a plugin uses config value A. A newer version, in order to support additional features, uses values B and C. It would be nice to be able to detect that neither B nor C were explicitly set by the administrator in records.config and therefore the plugin should fall back and use A. It is also required to *not* fall back to A if the administrator explicitly set B or C to the default value. Therefore checking the retrieved value against the default value is insufficient. One might use a bogus default value then, but now there's a problem if none of A,B, or C was set.

I don’t think this is sane :)

There’s no effective difference between a default value and a configured value. Mentioning a configuration value in `records.config` doesn’t mean “really make this so”, it just sets that value. So the newer plugin should just use B and C. Fallback to A would make sense if B and C are not present (eg. the value is a NULL string or some designared int), but this is specific to the plugin and unrelated to where the configuration value was set. Ad default is just as legitimate as an explicit configuration.

"It is also required to *not* fall back to A if the administrator explicitly set B or C to the default value”

This requirement leads to extremely confusing behavior and IMHO it is best to not go there.

Re: Config variable source values

Posted by James Peach <jp...@apache.org>.

> On Nov 21, 2016, at 1:22 PM, Alan Carroll <so...@yahoo-inc.com.INVALID> wrote:
> 
> The use case is providing backwards compatibility. Suppose a plugin uses config value A. A newer version, in order to support additional features, uses values B and C. It would be nice to be able to detect that neither B nor C were explicitly set by the administrator in records.config and therefore the plugin should fall back and use A. It is also required to *not* fall back to A if the administrator explicitly set B or C to the default value. Therefore checking the retrieved value against the default value is insufficient. One might use a bogus default value then, but now there's a problem if none of A,B, or C was set.

I don’t think this is sane :)

There’s no effective difference between a default value and a configured value. Mentioning a configuration value in `records.config` doesn’t mean “really make this so”, it just sets that value. So the newer plugin should just use B and C. Fallback to A would make sense if B and C are not present (eg. the value is a NULL string or some designared int), but this is specific to the plugin and unrelated to where the configuration value was set. Ad default is just as legitimate as an explicit configuration.

"It is also required to *not* fall back to A if the administrator explicitly set B or C to the default value”

This requirement leads to extremely confusing behavior and IMHO it is best to not go there.

>    On Monday, November 21, 2016 3:02 PM, Sudheer Vinukonda <su...@yahoo.com.INVALID> wrote:
> 
> 
> I'm curious to know the use case, if any, in where one'd care if a "default" value for a setting came from the source code or from records.config. 
> 
> Consequently, I wonder if it is sufficient to know that a given setting has a "default" value or a non-default value.
> 
> It might also be helpful if you can enumerate the possible combinations of values and the corresponding source/explicit value that they mean.
> 
> - Sudheer
> 
>> On Nov 21, 2016, at 12:52 PM, Alan Carroll <so...@yahoo-inc.com.INVALID> wrote:
>> 
>> No, they behave very differently. If you have a config variable in RecordsConfig.cc and it's not mentioned in records.config you get REC_SOURCE_DEFAULT. If you have a plugin created config variable that is not in records.config you get REC_SOURCE_EXPLICIT. In both cases if the value is in records.config you get REC_SOURCE_EXPLICIT. So for a built in it is detectable if the value is in records.config, but not detectable for a plugin created variable. 
>> 
>>     On Monday, November 21, 2016 2:34 PM, James Peach <jp...@apache.org> wrote:
>> 
>> 
>> 
>>> On Nov 21, 2016, at 11:50 AM, Alan Carroll <so...@yahoo-inc.com.INVALID> wrote:
>>> 
>>> No, the source is REC_SOURCE_EXPLICIT even if the variable is not mentioned in records.config. How else can a plugin tell if a config variable it creates has a value in records.config?
>> 
>> I think I’m confused about what you are proposing then. It sounds like configuration records used by plugins behave just like configuration records used by traffic_server. Is that the case? If so, it sounds like correct behavior to me. If not, what is the problem that you are addressing here?
>> 
>>>   
>>> 
>>>     On Monday, November 21, 2016 12:21 PM, James Peach <jp...@apache.org> wrote:
>>> 
>>> 
>>> 
>>>> On Nov 21, 2016, at 9:14 AM, Alan Carroll <so...@yahoo-inc.com.INVALID> wrote:
>>>> 
>>>> REC_SOURCE_EXPLICIT - see TSMgmtStringCreate in InkAPI.cc (around line 8896). That is the same value as setting the value in a configuration file.
>>> 
>>> That sounds correct to me. The value was set in a configuration file and is being consumed by a plugin.
>>> 
>>>>     On Monday, November 21, 2016 10:29 AM, James Peach <jp...@apache.org> wrote:
>>>> 
>>>> 
>>>> 
>>>>> On Nov 21, 2016, at 7:20 AM, Alan Carroll <so...@yahoo-inc.com.INVALID> wrote:
>>>>> 
>>>>> The config variable sourcing data was originally put in to be able to detect that a variable was set even if that setting was the same as the default (otherwise this is impossible to reliably determine). I am working with a plugin that defines its own config records but as far as I can tell (1) the source value for a plugin configured record default is the same as for an explicitly set value and (2) the source information is not available via the plugin API.
>>>> 
>>>> What is the source when you specify a record in records.config, then register it from a plugin?
>>>> 
>>>>> I would like to fix both of these.
>>>>> First, add a "PLUGIN" value to RecSourceT to indicate the value is set as a default by the plugin and use EXPLICIT only for values from configuration files or explicit value setting API calls.
>>>>> Second,
>>>>> * Move or copy the RecSourceT define to apidef.h.in.* Add TSMgmtSourceGet(const char*) to return the RecSourceT value for a configuration variable.
>>>>> 
>>>> 
>>>> 
>>> 
>>> 
>> 
>> 
> 
>

Re: Config variable source values

Posted by Sudheer Vinukonda <su...@yahoo.com.INVALID>.

Hmm..OK - Al though, I'm wondering this seems more of a band-aid approach to something that shouldn't happen (at least, not typically) and something that probably should be discouraged in principle? I fear It could get rather confusing and probably unpredictable if this sort of config setting changes are encouraged and supported more commonly.

- Sudheer

> On Nov 21, 2016, at 2:36 PM, Alan Carroll <so...@yahoo-inc.com.INVALID> wrote:
> 
> Sudheer - this was originally put in (TS-3650) precisely so we could add / change configuration values on point releases, so that in particular LTS releases could be tweaked to accept next major version configurations without breaking current major release configurations. This is quite handy if you're trying out the next major version vs. running on the LTS. The only real change I'm proposing here is making this already existing facility usable from plugins.
> James;
> "If `proxy.config.james` is set then the value causes blah blah blah behavior. If not set the deprecated configuration `proxy.config.peach` is used. If that is not set either, the default value "Sudheer" is used.".
> 
> 
>    On Monday, November 21, 2016 4:25 PM, Sudheer Vinukonda <su...@yahoo.com.INVALID> wrote:
> 
> 
> I see - so, IIUC, the problem you are trying to solve really is the *replacement* of an existing config setting with an altogether new config setting?
> 
> In that case, I suppose, the proposed additional info may be used *if* we are required to ensure backward compatibility, although, I'm not sure if that is a mandatory requirement across major release updates (unless, of course, we encounter a need to make this sort of change across a minor release update?).
> 
> 
> 
>> On Nov 21, 2016, at 2:18 PM, Alan Carroll <so...@yahoo-inc.com> wrote:
>> 
>> Sorry, A, B, and C are config variables, such as "proxy.config.quick_filter.mask", "proxy.config.quick_filter.mask_in", "proxy.config.quick_filter.mask_out".
>> 
>> 
>> On Monday, November 21, 2016 4:15 PM, Sudheer Vinukonda <su...@yahoo.com.INVALID> wrote:
>> 
>> 
>> Huh, I'm totally confused..Are "A", "B", "C" different config *settings* or different *values* for a *given* setting? Your original reply seemed to indicate (at least, the initial part of the reply) that they were *values* for a given setting, but, the latest seems to indicate the opposite :-/
>> "The use case is providing backwards compatibility. Suppose a plugin uses config value A. A newer version, in order to support additional features, uses values B and C. It would be nice to be able to detect that neither B nor C were explicitly set by the administrator in records.config and therefore the plugin should fall back and use A. It is also required to *not* fall back to A if the administrator explicitly set B or C to the default value. Therefore checking the retrieved value against the default value is insufficient. One might use a bogus default value then, but now there's a problem if none of A,B, or C was set."
>> 
>> 
>> PS: I am only trying to understand the use case or need/benefits of having this additional new info being proposed - not opposing the need for it!
>> 
>> 
>>       From: Alan Carroll <so...@yahoo-inc.com.INVALID>
>> 
>> To: "dev@trafficserver.apache.org" <de...@trafficserver.apache.org> 
>> Sent: Monday, November 21, 2016 1:55 PM
>> Subject: Re: Config variable source values
>>   
>> Let's say the default for B is "Sudheer". There are two cases I want to treat differently.
>> 1) No mention of B is in records.config. Fall back to A.2) In records.config, the value for B is set to "Sudheer" by the administrator. Use this value, do not look at A.
>> 
>> 
>>     On Monday, November 21, 2016 3:32 PM, Sudheer Vinukonda <su...@yahoo.com.INVALID> wrote:
>> 
>> 
>> Hmm..what do you mean by "the administrator explicitly set B or C to the default value"?
>> 
>>   
>> 
>>   
>> 
>> 
>

Re: Config variable source values

Posted by Sudheer Vinukonda <su...@yahoo.com.INVALID>.

The conflict is in the solution being proposed - 

AFAICT, merely not specifying a config setting explicitly in records.config does NOT mean, the resultant (default) behavior can be changed in a minor release update.

For example, take the case of server session sharing that was the origin of the whole discussion.

Let's say, 5.0 had a default of "global" server session sharing. Whether or not a specific deployment explicitly set server session sharing to global or not in records.config, the behavior they'd see is still global sharing (which has a certain noticeable external impact - example, number of origin connections, latency etc). 

Now, let's assume the default has been changed to "thread" pool sharing in 5.3. Simply because, some deployment did not specify explicitly that they wanted global sharing in records.config, they'd now start seeing a noticeable impact when updating to  5.3 because the default has been changed. To me, this clearly sounds like a "break" across minor update. It is at least not "seamless" as we claimed officially.

The right thing to do would be to maintain the default to global across all 5.x and if needed and agreed, the default can be changed in 6.0. The change in default would be documented in the release notes for 6.0 along with the expectation that when an administrator updates to 6.0, incompatible behavior changes are possible and the documentation provides the necessary warnings or mechanisms to address the consequences.

With the solution proposed, essentially, you are allowing any setting to change its default *anytime* and the only way one can guarantee the existing behavior is to manually/explicitly add the setting with the default in records.config even for minor release updates. 

This to me doesn't sound like a clean path (it's at least not what we claim officially as I read it).

Also, I still can't think of any other instances other than TS-3650 (which I argue is a direct consequence of a mere implementation bug) where this kind of change in default should be required or encouraged. Having this in core was probably not desirable to begin with, and now exposing it to plugins might make things even worse.

- Sudheer

> On Nov 22, 2016, at 8:50 AM, Alan Carroll <so...@yahoo-inc.com.INVALID> wrote:
> 
> I don't see the conflict with the policy you quoted. Use of the config variable source mechanism means that new configuration options can be added to minor versions because it can be done without an incompatible change, exactly in line with "MINOR version when you add functionality in a backwards-compatible manner". I think the ability to have a smoother transition, where configuration changes are enabled but not required, is very nice for the people who actually deploy ATS.
> But the real local issue here is, should we have the TS-3650 mechanism only for core and not for plugins? What is the justification for having it not work only for plugins?
> 
>

Re: Config variable source values

Posted by Alan Carroll <so...@yahoo-inc.com.INVALID>.

 I don't see the conflict with the policy you quoted. Use of the config variable source mechanism means that new configuration options can be added to minor versions because it can be done without an incompatible change, exactly in line with "MINOR version when you add functionality in a backwards-compatible manner". I think the ability to have a smoother transition, where configuration changes are enabled but not required, is very nice for the people who actually deploy ATS.
But the real local issue here is, should we have the TS-3650 mechanism only for core and not for plugins? What is the justification for having it not work only for plugins?

Re: Config variable source values

Posted by Sudheer Vinukonda <su...@yahoo.com.INVALID>.

Here&#39;s what curl says about its upgrade policy -

https://curl.haxx.se/libcurl/abi.html

&quot;In the vast majority of all cases, a typical libcurl upgrade does not break the ABI at all. Your application can remain using libcurl just as before, only with less bugs and possibly with added new features. You need to read the release notes, and if they mention an ABI break/soname bump, you may have to verify that your application still builds fine and uses libcurl as it now is defined to work.&quot;

Here&#39;s what Trafiicserver says - 

https://docs.trafficserver.apache.org/en/latest/admin-guide/installation/index.en.html?highlight=compatibility

&quot;Traffic Server Versioning
Before you get started with Traffic Server you may have to decide which version you want to use. Traffic Server follows the Semantic Versioning guidelines.&quot;

&quot;http://semver.org/

Given a version number MAJOR.MINOR.PATCH, increment the:

MAJOR version when you make incompatible API changes,
MINOR version when you add functionality in a backwards-compatible manner, and
PATCH version when you make backwards-compatible bug fixes.
Additional labels for pre-release and build metadata are available as extensions to the MAJOR.MINOR.PATCH format.&quot;


IMHO, we should try to stick to our policies for version updates as far as possible and avoid unnncessary complexities or inconsistent/non-uniform behavior to the one we claim officially. 

I still think TS-3650 was a result of an implementation error that unfortunately overlooked backward compatibility and resulted in a breakage across minor version update. And to my knowledge, there really was no intention to break it on purpose nor there has been another instance where this happened. IOW, I view TS-3650/TS-3642 etc as simple bug fixes and something that should not recur.

We have generally been very diligent and successful at discouraging when proposals to change defaults or behavior within a major release come along. I see no reason to go back on that policy purely based on one erroneous implementation (which could have easily been reversed rather than invent another mechanism to seemingly encourage that behavior more consistently). And if we did, it probably is a much larger scope discussion and may involve other things besides config default changes.

You mentioned adoption of new major version within an LTS release as a reason to support this functionality - I am not sure that&#39;s a strong enough justification, since generally, the new default can likely be achieved by overriding the default in LTS already (otherwise, the functionality that the new default provides is not possible at all in the LTS release).

Again, I think all of this comes down to whether we want to encourage this sort of behavior changes (changing defaults) across minor updates and I&#39;d think not and it can wait for the next major update typically. 

- Sudheer

Re: Config variable source values

Posted by Alan Carroll <so...@yahoo-inc.com.INVALID>.

Unfortunately it's not so far fetched that I don't need it to implement required support in a plugin, and not so far fetched I haven't had to do it at least twice in the core already.

Re: Config variable source values

Posted by Sudheer Vinukonda <su...@yahoo.com.INVALID>.

To avoid going in circles and to summarize my 2c, I've to say I agree with James that the whole proposal and the problem description looks rather unconvincing and extraneous at best. Be so it may, extending this capability to plugins seems even more far-fetched.

- Sudheer

> On Nov 22, 2016, at 12:23 PM, Alan Carroll <so...@yahoo-inc.com.INVALID> wrote:
> 
> In order to enable ops to experiment/test/verify the new configuration options before upgrading to the next major version, without breaking other people.
> Note also this isn't a *replace*, it's an augmentation. The whole point is to enable existing configuration to continue to work exactly as it did before.
> If it is to be a replace across major versions this enables operations to verify the new configuration options independently of upgrading to the new version. The goal is to ease the pain of transition to the next major version.
> 
>    On Tuesday, November 22, 2016 1:36 PM, Sudheer Vinukonda <su...@yahoo.com.INVALID> wrote:
> 
> 
> But, practically, on purpose, why'd you ever want to replace an existing config setting with another new config setting in a minor release (I'm not referring to implementation bugs)? That sort of change should belong to a major release. The argument that it makes it easier to inherit new major release features into LTS doesn't sound to me as strong enough reason, simply because, presumably the new feature's default functionality should be possible with the updating the older setting to a corresponding non-default value in the LTS version already.
> 
>> On Nov 22, 2016, at 9:34 AM, Alan Carroll <so...@yahoo-inc.com.INVALID> wrote:
>> 
>> Sudheer;
>> "Now, let's assume the default has been changed to "thread" pool sharing in 5.3".
>> No, let's not. That's the opposite of what I'm talking about. The entire point of all of this is to *NOT* change the default behavior. In fact, the goal is to not change behavior at all unless the administrator specifically and explicitly uses the new configuration variables. The reason the default changed was because I thought the TS-3650 mechanism already existed (e.g., it was detectable if the administrator used the new configuration variables or not). Adding that mechanism is how to avoid changing the default value from "global" to "thread".
>> The summary is, I want to be able to experiment / add new configuration without changing the behavior of any existing configuration file. TS-3650 lets me do that in core, I want to make some minor changes to enable this in a plugin as well.
>> 
>> Another example would be the change in how server ports are defined. The prior version used two different configuration variables while the current version uses just one. However, while this was changing it had no effect on existing configurations, which continued to behave the same as before the change. It would have been much easier to do that with the TS-3650 mechanism.
>> James;
>> I need this change to keep compatibility. So, since you're in favor of that, I'll take it you're OK with this change.
>> 
>> 
> 
>

Re: Config variable source values

Posted by Alan Carroll <so...@yahoo-inc.com.INVALID>.

In order to enable ops to experiment/test/verify the new configuration options before upgrading to the next major version, without breaking other people.
Note also this isn't a *replace*, it's an augmentation. The whole point is to enable existing configuration to continue to work exactly as it did before.
If it is to be a replace across major versions this enables operations to verify the new configuration options independently of upgrading to the new version. The goal is to ease the pain of transition to the next major version.

    On Tuesday, November 22, 2016 1:36 PM, Sudheer Vinukonda <su...@yahoo.com.INVALID> wrote:

 But, practically, on purpose, why'd you ever want to replace an existing config setting with another new config setting in a minor release (I'm not referring to implementation bugs)? That sort of change should belong to a major release. The argument that it makes it easier to inherit new major release features into LTS doesn't sound to me as strong enough reason, simply because, presumably the new feature's default functionality should be possible with the updating the older setting to a corresponding non-default value in the LTS version already.

> On Nov 22, 2016, at 9:34 AM, Alan Carroll <so...@yahoo-inc.com.INVALID> wrote:
> 
> Sudheer;
> "Now, let's assume the default has been changed to "thread" pool sharing in 5.3".
> No, let's not. That's the opposite of what I'm talking about. The entire point of all of this is to *NOT* change the default behavior. In fact, the goal is to not change behavior at all unless the administrator specifically and explicitly uses the new configuration variables. The reason the default changed was because I thought the TS-3650 mechanism already existed (e.g., it was detectable if the administrator used the new configuration variables or not). Adding that mechanism is how to avoid changing the default value from "global" to "thread".
> The summary is, I want to be able to experiment / add new configuration without changing the behavior of any existing configuration file. TS-3650 lets me do that in core, I want to make some minor changes to enable this in a plugin as well.
> 
> Another example would be the change in how server ports are defined. The prior version used two different configuration variables while the current version uses just one. However, while this was changing it had no effect on existing configurations, which continued to behave the same as before the change. It would have been much easier to do that with the TS-3650 mechanism.
> James;
> I need this change to keep compatibility. So, since you're in favor of that, I'll take it you're OK with this change.
> 
>

Re: Config variable source values

Posted by Sudheer Vinukonda <su...@yahoo.com.INVALID>.

But, practically, on purpose, why'd you ever want to replace an existing config setting with another new config setting in a minor release (I'm not referring to implementation bugs)? That sort of change should belong to a major release. The argument that it makes it easier to inherit new major release features into LTS doesn't sound to me as strong enough reason, simply because, presumably the new feature's default functionality should be possible with the updating the older setting to a corresponding non-default value in the LTS version already.

> On Nov 22, 2016, at 9:34 AM, Alan Carroll <so...@yahoo-inc.com.INVALID> wrote:
> 
> Sudheer;
> "Now, let's assume the default has been changed to "thread" pool sharing in 5.3".
> No, let's not. That's the opposite of what I'm talking about. The entire point of all of this is to *NOT* change the default behavior. In fact, the goal is to not change behavior at all unless the administrator specifically and explicitly uses the new configuration variables. The reason the default changed was because I thought the TS-3650 mechanism already existed (e.g., it was detectable if the administrator used the new configuration variables or not). Adding that mechanism is how to avoid changing the default value from "global" to "thread".
> The summary is, I want to be able to experiment / add new configuration without changing the behavior of any existing configuration file. TS-3650 lets me do that in core, I want to make some minor changes to enable this in a plugin as well.
> 
> Another example would be the change in how server ports are defined. The prior version used two different configuration variables while the current version uses just one. However, while this was changing it had no effect on existing configurations, which continued to behave the same as before the change. It would have been much easier to do that with the TS-3650 mechanism.
> James;
> I need this change to keep compatibility. So, since you're in favor of that, I'll take it you're OK with this change.
> 
>

Re: Config variable source values

Posted by Alan Carroll <so...@yahoo-inc.com.INVALID>.

Sudheer;
"Now, let's assume the default has been changed to "thread" pool sharing in 5.3".
No, let's not. That's the opposite of what I'm talking about. The entire point of all of this is to *NOT* change the default behavior. In fact, the goal is to not change behavior at all unless the administrator specifically and explicitly uses the new configuration variables. The reason the default changed was because I thought the TS-3650 mechanism already existed (e.g., it was detectable if the administrator used the new configuration variables or not). Adding that mechanism is how to avoid changing the default value from "global" to "thread".
The summary is, I want to be able to experiment / add new configuration without changing the behavior of any existing configuration file. TS-3650 lets me do that in core, I want to make some minor changes to enable this in a plugin as well.

Another example would be the change in how server ports are defined. The prior version used two different configuration variables while the current version uses just one. However, while this was changing it had no effect on existing configurations, which continued to behave the same as before the change. It would have been much easier to do that with the TS-3650 mechanism.
James;
I need this change to keep compatibility. So, since you're in favor of that, I'll take it you're OK with this change.

Re: Config variable source values

Posted by Jason Kenny <jk...@yahoo-inc.com.INVALID>.

I have to say this is an odd argument.
Anyone that does real world code in enterprise environments knows that you have to migrate between versions. Config value that are renamed have to allow for time migrate to a new value. Hard switches will upset customers very quickly. They need time to be warned to move. Any quality configuration systems has a notion of is the value defined, is there some sort of fall back mapping, and maybe a defined default ( as in the case of ATS) that is used if there is no value defined and no fall back mapping of a deprecated setting. Values take time to be removed. They happen at the start of a Major version release, never on a B or C version level change. They need at least on Major (A version) to be deprecated before they can be removed. I think the only exception to this is that a value was added as experimental and was never made official/stable. ( ie nobody should be using it in production)
just my quick 2 cents

Jason

On Tuesday, November 22, 2016, 11:13:42 AM CST, James Peach <jp...@apache.org> wrote:
> On Nov 22, 2016, at 6:25 AM, Alan Carroll <so...@yahoo-inc.com.INVALID> wrote:
> 
> "If `proxy.config.james` supersedes the old configuration, I should not have to explicitly specify it in records.config for that to work"
> Correct. And if the administrator specifies no configuration variable, that's what happens. If the administrator does specify it, then it works. The case of interest is where the administrator does NOT specify the new configuration value and DOES specify the old one. In that case I think it's clear the administrator expects the old setting to work. Your argument is the administrator did that in order to get the default behavior of the new configuration setting?

No, my argument is that you can’t read too much into this. If you want to keep compatibility, great, keep it. You can’t tell whether an operator depends on a config value by looking at whether they used the default or specified it in records.config.

> My experience in actually deploying stuff and working with those who do is this sort of backwards compatibility is highly valued. It makes rolling out new versions much easier. It also seems a common thing to do - look at curl parameters for instance, where old switches are maintained while providing newer equivalent ones.

This is simply normal backwards compatibility practice. Keep reading the old configurations around forever and don’t change their semantics. If the parameter was actually removed then there is no default, and you can just test for its presence (maybe this is similar to the change you are proposing?)

> It also doesn't seem different from the discussions about configuration file formats and maintaining the ability to read old formats even after adding the new format.

I specifically asked this about logging.config and no-one cared. This did surprise me since I expect that most people do change the logging format.

I agree with you that we can do a better job of making upgrades. Maybe an explicit upgrade tool or configuration phase that knows how to convert configuration from one release to the next?

J

Re: Config variable source values

Posted by James Peach <jp...@apache.org>.

> On Nov 22, 2016, at 6:25 AM, Alan Carroll <so...@yahoo-inc.com.INVALID> wrote:
> 
> "If `proxy.config.james` supersedes the old configuration, I should not have to explicitly specify it in records.config for that to work"
> Correct. And if the administrator specifies no configuration variable, that's what happens. If the administrator does specify it, then it works. The case of interest is where the administrator does NOT specify the new configuration value and DOES specify the old one. In that case I think it's clear the administrator expects the old setting to work. Your argument is the administrator did that in order to get the default behavior of the new configuration setting?

No, my argument is that you can’t read too much into this. If you want to keep compatibility, great, keep it. You can’t tell whether an operator depends on a config value by looking at whether they used the default or specified it in records.config.

> My experience in actually deploying stuff and working with those who do is this sort of backwards compatibility is highly valued. It makes rolling out new versions much easier. It also seems a common thing to do - look at curl parameters for instance, where old switches are maintained while providing newer equivalent ones.

This is simply normal backwards compatibility practice. Keep reading the old configurations around forever and don’t change their semantics. If the parameter was actually removed then there is no default, and you can just test for its presence (maybe this is similar to the change you are proposing?)

> It also doesn't seem different from the discussions about configuration file formats and maintaining the ability to read old formats even after adding the new format.

I specifically asked this about logging.config and no-one cared. This did surprise me since I expect that most people do change the logging format.

I agree with you that we can do a better job of making upgrades. Maybe an explicit upgrade tool or configuration phase that knows how to convert configuration from one release to the next?

J

Re: Config variable source values

Posted by Alan Carroll <so...@yahoo-inc.com.INVALID>.

"If `proxy.config.james` supersedes the old configuration, I should not have to explicitly specify it in records.config for that to work"
Correct. And if the administrator specifies no configuration variable, that's what happens. If the administrator does specify it, then it works. The case of interest is where the administrator does NOT specify the new configuration value and DOES specify the old one. In that case I think it's clear the administrator expects the old setting to work. Your argument is the administrator did that in order to get the default behavior of the new configuration setting?

My experience in actually deploying stuff and working with those who do is this sort of backwards compatibility is highly valued. It makes rolling out new versions much easier. It also seems a common thing to do - look at curl parameters for instance, where old switches are maintained while providing newer equivalent ones. It also doesn't seem different from the discussions about configuration file formats and maintaining the ability to read old formats even after adding the new format.

Re: Config variable source values

Posted by James Peach <jp...@apache.org>.

> On Nov 21, 2016, at 2:36 PM, Alan Carroll <so...@yahoo-inc.com.INVALID> wrote:
> 
> Sudheer - this was originally put in (TS-3650) precisely so we could add / change configuration values on point releases, so that in particular LTS releases could be tweaked to accept next major version configurations without breaking current major release configurations. This is quite handy if you're trying out the next major version vs. running on the LTS. The only real change I'm proposing here is making this already existing facility usable from plugins.
> James;
> "If `proxy.config.james` is set then the value causes blah blah blah behavior. If not set the deprecated configuration `proxy.config.peach` is used. If that is not set either, the default value "Sudheer" is used.”.

If `proxy.config.james` supersedes the old configuration, I should not have to explicitly specify it in records.config for that to work. Why shouldn’t I be able to depend on the documented defaults like I can with every other configuration? What happens if they are both specified in records.config? What happens if `proxy.config.james` wasn’t specified in records.config and then it is after a reload?

I’m still unconvinced :)


>    On Monday, November 21, 2016 4:25 PM, Sudheer Vinukonda <su...@yahoo.com.INVALID> wrote:
> 
> 
> I see - so, IIUC, the problem you are trying to solve really is the *replacement* of an existing config setting with an altogether new config setting?
> 
> In that case, I suppose, the proposed additional info may be used *if* we are required to ensure backward compatibility, although, I'm not sure if that is a mandatory requirement across major release updates (unless, of course, we encounter a need to make this sort of change across a minor release update?).
> 
> 
> 
>> On Nov 21, 2016, at 2:18 PM, Alan Carroll <so...@yahoo-inc.com> wrote:
>> 
>> Sorry, A, B, and C are config variables, such as "proxy.config.quick_filter.mask", "proxy.config.quick_filter.mask_in", "proxy.config.quick_filter.mask_out".
>> 
>> 
>> On Monday, November 21, 2016 4:15 PM, Sudheer Vinukonda <su...@yahoo.com.INVALID> wrote:
>> 
>> 
>> Huh, I'm totally confused..Are "A", "B", "C" different config *settings* or different *values* for a *given* setting? Your original reply seemed to indicate (at least, the initial part of the reply) that they were *values* for a given setting, but, the latest seems to indicate the opposite :-/
>> "The use case is providing backwards compatibility. Suppose a plugin uses config value A. A newer version, in order to support additional features, uses values B and C. It would be nice to be able to detect that neither B nor C were explicitly set by the administrator in records.config and therefore the plugin should fall back and use A. It is also required to *not* fall back to A if the administrator explicitly set B or C to the default value. Therefore checking the retrieved value against the default value is insufficient. One might use a bogus default value then, but now there's a problem if none of A,B, or C was set."
>> 
>> 
>> PS: I am only trying to understand the use case or need/benefits of having this additional new info being proposed - not opposing the need for it!
>> 
>> 
>>       From: Alan Carroll <so...@yahoo-inc.com.INVALID>
>> 
>> To: "dev@trafficserver.apache.org" <de...@trafficserver.apache.org> 
>> Sent: Monday, November 21, 2016 1:55 PM
>> Subject: Re: Config variable source values
>>   
>> Let's say the default for B is "Sudheer". There are two cases I want to treat differently.
>> 1) No mention of B is in records.config. Fall back to A.2) In records.config, the value for B is set to "Sudheer" by the administrator. Use this value, do not look at A.
>> 
>> 
>>     On Monday, November 21, 2016 3:32 PM, Sudheer Vinukonda <su...@yahoo.com.INVALID> wrote:
>> 
>> 
>> Hmm..what do you mean by "the administrator explicitly set B or C to the default value"?
>> 
>>   
>> 
>>   
>> 
>> 
>

Re: Config variable source values

Posted by Alan Carroll <so...@yahoo-inc.com.INVALID>.

Sudheer - this was originally put in (TS-3650) precisely so we could add / change configuration values on point releases, so that in particular LTS releases could be tweaked to accept next major version configurations without breaking current major release configurations. This is quite handy if you're trying out the next major version vs. running on the LTS. The only real change I'm proposing here is making this already existing facility usable from plugins.
James;
"If `proxy.config.james` is set then the value causes blah blah blah behavior. If not set the deprecated configuration `proxy.config.peach` is used. If that is not set either, the default value "Sudheer" is used.".

    On Monday, November 21, 2016 4:25 PM, Sudheer Vinukonda <su...@yahoo.com.INVALID> wrote:

 I see - so, IIUC, the problem you are trying to solve really is the *replacement* of an existing config setting with an altogether new config setting?

In that case, I suppose, the proposed additional info may be used *if* we are required to ensure backward compatibility, although, I'm not sure if that is a mandatory requirement across major release updates (unless, of course, we encounter a need to make this sort of change across a minor release update?).

> On Nov 21, 2016, at 2:18 PM, Alan Carroll <so...@yahoo-inc.com> wrote:
> 
> Sorry, A, B, and C are config variables, such as "proxy.config.quick_filter.mask", "proxy.config.quick_filter.mask_in", "proxy.config.quick_filter.mask_out".
> 
> 
> On Monday, November 21, 2016 4:15 PM, Sudheer Vinukonda <su...@yahoo.com.INVALID> wrote:
> 
> 
> Huh, I'm totally confused..Are "A", "B", "C" different config *settings* or different *values* for a *given* setting? Your original reply seemed to indicate (at least, the initial part of the reply) that they were *values* for a given setting, but, the latest seems to indicate the opposite :-/
> "The use case is providing backwards compatibility. Suppose a plugin uses config value A. A newer version, in order to support additional features, uses values B and C. It would be nice to be able to detect that neither B nor C were explicitly set by the administrator in records.config and therefore the plugin should fall back and use A. It is also required to *not* fall back to A if the administrator explicitly set B or C to the default value. Therefore checking the retrieved value against the default value is insufficient. One might use a bogus default value then, but now there's a problem if none of A,B, or C was set."
> 
> 
> PS: I am only trying to understand the use case or need/benefits of having this additional new info being proposed - not opposing the need for it!
> 
> 
>      From: Alan Carroll <so...@yahoo-inc.com.INVALID>
> 
> To: "dev@trafficserver.apache.org" <de...@trafficserver.apache.org> 
> Sent: Monday, November 21, 2016 1:55 PM
> Subject: Re: Config variable source values
>  
> Let's say the default for B is "Sudheer". There are two cases I want to treat differently.
> 1) No mention of B is in records.config. Fall back to A.2) In records.config, the value for B is set to "Sudheer" by the administrator. Use this value, do not look at A.
> 
> 
>    On Monday, November 21, 2016 3:32 PM, Sudheer Vinukonda <su...@yahoo.com.INVALID> wrote:
> 
> 
> Hmm..what do you mean by "the administrator explicitly set B or C to the default value"?
> 
>  
> 
>  
> 
>

Re: Config variable source values

Posted by Sudheer Vinukonda <su...@yahoo.com.INVALID>.

I see - so, IIUC, the problem you are trying to solve really is the *replacement* of an existing config setting with an altogether new config setting?

In that case, I suppose, the proposed additional info may be used *if* we are required to ensure backward compatibility, although, I'm not sure if that is a mandatory requirement across major release updates (unless, of course, we encounter a need to make this sort of change across a minor release update?).



> On Nov 21, 2016, at 2:18 PM, Alan Carroll <so...@yahoo-inc.com> wrote:
> 
> Sorry, A, B, and C are config variables, such as "proxy.config.quick_filter.mask", "proxy.config.quick_filter.mask_in", "proxy.config.quick_filter.mask_out".
> 
> 
> On Monday, November 21, 2016 4:15 PM, Sudheer Vinukonda <su...@yahoo.com.INVALID> wrote:
> 
> 
> Huh, I'm totally confused..Are "A", "B", "C" different config *settings* or different *values* for a *given* setting? Your original reply seemed to indicate (at least, the initial part of the reply) that they were *values* for a given setting, but, the latest seems to indicate the opposite :-/
> "The use case is providing backwards compatibility. Suppose a plugin uses config value A. A newer version, in order to support additional features, uses values B and C. It would be nice to be able to detect that neither B nor C were explicitly set by the administrator in records.config and therefore the plugin should fall back and use A. It is also required to *not* fall back to A if the administrator explicitly set B or C to the default value. Therefore checking the retrieved value against the default value is insufficient. One might use a bogus default value then, but now there's a problem if none of A,B, or C was set."
> 
> 
> PS: I am only trying to understand the use case or need/benefits of having this additional new info being proposed - not opposing the need for it!
> 
> 
>       From: Alan Carroll <so...@yahoo-inc.com.INVALID>
> 
> To: "dev@trafficserver.apache.org" <de...@trafficserver.apache.org> 
> Sent: Monday, November 21, 2016 1:55 PM
> Subject: Re: Config variable source values
>   
> Let's say the default for B is "Sudheer". There are two cases I want to treat differently.
> 1) No mention of B is in records.config. Fall back to A.2) In records.config, the value for B is set to "Sudheer" by the administrator. Use this value, do not look at A.
> 
> 
>     On Monday, November 21, 2016 3:32 PM, Sudheer Vinukonda <su...@yahoo.com.INVALID> wrote:
> 
> 
> Hmm..what do you mean by "the administrator explicitly set B or C to the default value"?
> 
>   
> 
>  
> 
>

Re: Config variable source values

Posted by Alan Carroll <so...@yahoo-inc.com.INVALID>.

Sorry, A, B, and C are config variables, such as "proxy.config.quick_filter.mask", "proxy.config.quick_filter.mask_in", "proxy.config.quick_filter.mask_out".

On Monday, November 21, 2016 4:15 PM, Sudheer Vinukonda <su...@yahoo.com.INVALID> wrote:

Huh, I'm totally confused..Are "A", "B", "C" different config *settings* or different *values* for a *given* setting? Your original reply seemed to indicate (at least, the initial part of the reply) that they were *values* for a given setting, but, the latest seems to indicate the opposite :-/
"The use case is providing backwards compatibility. Suppose a plugin uses config value A. A newer version, in order to support additional features, uses values B and C. It would be nice to be able to detect that neither B nor C were explicitly set by the administrator in records.config and therefore the plugin should fall back and use A. It is also required to *not* fall back to A if the administrator explicitly set B or C to the default value. Therefore checking the retrieved value against the default value is insufficient. One might use a bogus default value then, but now there's a problem if none of A,B, or C was set."

PS: I am only trying to understand the use case or need/benefits of having this additional new info being proposed - not opposing the need for it!

From: Alan Carroll <so...@yahoo-inc.com.INVALID>
To: "dev@trafficserver.apache.org" <de...@trafficserver.apache.org>
Sent: Monday, November 21, 2016 1:55 PM
Subject: Re: Config variable source values
Let's say the default for B is "Sudheer". There are two cases I want to treat differently.
1) No mention of B is in records.config. Fall back to A.2) In records.config, the value for B is set to "Sudheer" by the administrator. Use this value, do not look at A.

On Monday, November 21, 2016 3:32 PM, Sudheer Vinukonda <su...@yahoo.com.INVALID> wrote:

Hmm..what do you mean by "the administrator explicitly set B or C to the default value"?

Re: Config variable source values

Posted by Sudheer Vinukonda <su...@yahoo.com.INVALID>.

PS: I am only trying to understand the use case or need/benefits of having this additional new info being proposed - not opposing the need for it!

From: Alan Carroll <so...@yahoo-inc.com.INVALID>
To: "dev@trafficserver.apache.org" <de...@trafficserver.apache.org>
Sent: Monday, November 21, 2016 1:55 PM
Subject: Re: Config variable source values

Let's say the default for B is "Sudheer". There are two cases I want to treat differently.
1) No mention of B is in records.config. Fall back to A.2) In records.config, the value for B is set to "Sudheer" by the administrator. Use this value, do not look at A.

On Monday, November 21, 2016 3:32 PM, Sudheer Vinukonda <su...@yahoo.com.INVALID> wrote:

Hmm..what do you mean by "the administrator explicitly set B or C to the default value"?

Re: Config variable source values

Posted by Alan Carroll <so...@yahoo-inc.com.INVALID>.

Let's say the default for B is "Sudheer". There are two cases I want to treat differently.
1) No mention of B is in records.config. Fall back to A.2) In records.config, the value for B is set to "Sudheer" by the administrator. Use this value, do not look at A.
 

    On Monday, November 21, 2016 3:32 PM, Sudheer Vinukonda <su...@yahoo.com.INVALID> wrote:
 

 Hmm..what do you mean by "the administrator explicitly set B or C to the default value"?

Re: Config variable source values

Posted by Sudheer Vinukonda <su...@yahoo.com.INVALID>.

Hmm..what do you mean by "the administrator explicitly set B or C to the default value"?

I'm not sure how the administrator can "play" around with default values? Or Did you mean, if a new version (release?) of trafficserver changed the default value for a setting?

If it is the latter (changing default across releases), shouldn't our normal guidelines/warnings about compatibility across releases (generally, backward compatible across minor version updates, possibly not backward compatible across major updates) handle the consequences? 



> On Nov 21, 2016, at 1:22 PM, Alan Carroll <so...@yahoo-inc.com.INVALID> wrote:
> 
> The use case is providing backwards compatibility. Suppose a plugin uses config value A. A newer version, in order to support additional features, uses values B and C. It would be nice to be able to detect that neither B nor C were explicitly set by the administrator in records.config and therefore the plugin should fall back and use A. It is also required to *not* fall back to A if the administrator explicitly set B or C to the default value. Therefore checking the retrieved value against the default value is insufficient. One might use a bogus default value then, but now there's a problem if none of A,B, or C was set.
> 
> 
> 
>    On Monday, November 21, 2016 3:02 PM, Sudheer Vinukonda <su...@yahoo.com.INVALID> wrote:
> 
> 
> I'm curious to know the use case, if any, in where one'd care if a "default" value for a setting came from the source code or from records.config. 
> 
> Consequently, I wonder if it is sufficient to know that a given setting has a "default" value or a non-default value.
> 
> It might also be helpful if you can enumerate the possible combinations of values and the corresponding source/explicit value that they mean.
> 
> - Sudheer
> 
>> On Nov 21, 2016, at 12:52 PM, Alan Carroll <so...@yahoo-inc.com.INVALID> wrote:
>> 
>> No, they behave very differently. If you have a config variable in RecordsConfig.cc and it's not mentioned in records.config you get REC_SOURCE_DEFAULT. If you have a plugin created config variable that is not in records.config you get REC_SOURCE_EXPLICIT. In both cases if the value is in records.config you get REC_SOURCE_EXPLICIT. So for a built in it is detectable if the value is in records.config, but not detectable for a plugin created variable. 
>> 
>>     On Monday, November 21, 2016 2:34 PM, James Peach <jp...@apache.org> wrote:
>> 
>> 
>> 
>>> On Nov 21, 2016, at 11:50 AM, Alan Carroll <so...@yahoo-inc.com.INVALID> wrote:
>>> 
>>> No, the source is REC_SOURCE_EXPLICIT even if the variable is not mentioned in records.config. How else can a plugin tell if a config variable it creates has a value in records.config?
>> 
>> I think I’m confused about what you are proposing then. It sounds like configuration records used by plugins behave just like configuration records used by traffic_server. Is that the case? If so, it sounds like correct behavior to me. If not, what is the problem that you are addressing here?
>> 
>>>   
>>> 
>>>     On Monday, November 21, 2016 12:21 PM, James Peach <jp...@apache.org> wrote:
>>> 
>>> 
>>> 
>>>> On Nov 21, 2016, at 9:14 AM, Alan Carroll <so...@yahoo-inc.com.INVALID> wrote:
>>>> 
>>>> REC_SOURCE_EXPLICIT - see TSMgmtStringCreate in InkAPI.cc (around line 8896). That is the same value as setting the value in a configuration file.
>>> 
>>> That sounds correct to me. The value was set in a configuration file and is being consumed by a plugin.
>>> 
>>>>     On Monday, November 21, 2016 10:29 AM, James Peach <jp...@apache.org> wrote:
>>>> 
>>>> 
>>>> 
>>>>> On Nov 21, 2016, at 7:20 AM, Alan Carroll <so...@yahoo-inc.com.INVALID> wrote:
>>>>> 
>>>>> The config variable sourcing data was originally put in to be able to detect that a variable was set even if that setting was the same as the default (otherwise this is impossible to reliably determine). I am working with a plugin that defines its own config records but as far as I can tell (1) the source value for a plugin configured record default is the same as for an explicitly set value and (2) the source information is not available via the plugin API.
>>>> 
>>>> What is the source when you specify a record in records.config, then register it from a plugin?
>>>> 
>>>>> I would like to fix both of these.
>>>>> First, add a "PLUGIN" value to RecSourceT to indicate the value is set as a default by the plugin and use EXPLICIT only for values from configuration files or explicit value setting API calls.
>>>>> Second,
>>>>> * Move or copy the RecSourceT define to apidef.h.in.* Add TSMgmtSourceGet(const char*) to return the RecSourceT value for a configuration variable.
>>>>> 
>>>> 
>>>> 
>>> 
>>> 
>> 
>> 
> 
>

Re: Config variable source values

Posted by Alan Carroll <so...@yahoo-inc.com.INVALID>.

The use case is providing backwards compatibility. Suppose a plugin uses config value A. A newer version, in order to support additional features, uses values B and C. It would be nice to be able to detect that neither B nor C were explicitly set by the administrator in records.config and therefore the plugin should fall back and use A. It is also required to *not* fall back to A if the administrator explicitly set B or C to the default value. Therefore checking the retrieved value against the default value is insufficient. One might use a bogus default value then, but now there's a problem if none of A,B, or C was set.

 

    On Monday, November 21, 2016 3:02 PM, Sudheer Vinukonda <su...@yahoo.com.INVALID> wrote:
 

 I'm curious to know the use case, if any, in where one'd care if a "default" value for a setting came from the source code or from records.config. 

Consequently, I wonder if it is sufficient to know that a given setting has a "default" value or a non-default value.

It might also be helpful if you can enumerate the possible combinations of values and the corresponding source/explicit value that they mean.

- Sudheer

> On Nov 21, 2016, at 12:52 PM, Alan Carroll <so...@yahoo-inc.com.INVALID> wrote:
> 
> No, they behave very differently. If you have a config variable in RecordsConfig.cc and it's not mentioned in records.config you get REC_SOURCE_DEFAULT. If you have a plugin created config variable that is not in records.config you get REC_SOURCE_EXPLICIT. In both cases if the value is in records.config you get REC_SOURCE_EXPLICIT. So for a built in it is detectable if the value is in records.config, but not detectable for a plugin created variable. 
> 
>    On Monday, November 21, 2016 2:34 PM, James Peach <jp...@apache.org> wrote:
> 
> 
> 
>> On Nov 21, 2016, at 11:50 AM, Alan Carroll <so...@yahoo-inc.com.INVALID> wrote:
>> 
>> No, the source is REC_SOURCE_EXPLICIT even if the variable is not mentioned in records.config. How else can a plugin tell if a config variable it creates has a value in records.config?
> 
> I think I’m confused about what you are proposing then. It sounds like configuration records used by plugins behave just like configuration records used by traffic_server. Is that the case? If so, it sounds like correct behavior to me. If not, what is the problem that you are addressing here?
> 
>>  
>> 
>>    On Monday, November 21, 2016 12:21 PM, James Peach <jp...@apache.org> wrote:
>> 
>> 
>> 
>>> On Nov 21, 2016, at 9:14 AM, Alan Carroll <so...@yahoo-inc.com.INVALID> wrote:
>>> 
>>> REC_SOURCE_EXPLICIT - see TSMgmtStringCreate in InkAPI.cc (around line 8896). That is the same value as setting the value in a configuration file.
>> 
>> That sounds correct to me. The value was set in a configuration file and is being consumed by a plugin.
>> 
>>>    On Monday, November 21, 2016 10:29 AM, James Peach <jp...@apache.org> wrote:
>>> 
>>> 
>>> 
>>>> On Nov 21, 2016, at 7:20 AM, Alan Carroll <so...@yahoo-inc.com.INVALID> wrote:
>>>> 
>>>> The config variable sourcing data was originally put in to be able to detect that a variable was set even if that setting was the same as the default (otherwise this is impossible to reliably determine). I am working with a plugin that defines its own config records but as far as I can tell (1) the source value for a plugin configured record default is the same as for an explicitly set value and (2) the source information is not available via the plugin API.
>>> 
>>> What is the source when you specify a record in records.config, then register it from a plugin?
>>> 
>>>> I would like to fix both of these.
>>>> First, add a "PLUGIN" value to RecSourceT to indicate the value is set as a default by the plugin and use EXPLICIT only for values from configuration files or explicit value setting API calls.
>>>> Second,
>>>> * Move or copy the RecSourceT define to apidef.h.in.* Add TSMgmtSourceGet(const char*) to return the RecSourceT value for a configuration variable.
>>>> 
>>> 
>>> 
>> 
>> 
> 
>

Re: Config variable source values

Posted by Sudheer Vinukonda <su...@yahoo.com.INVALID>.

I'm curious to know the use case, if any, in where one'd care if a "default" value for a setting came from the source code or from records.config. 

Consequently, I wonder if it is sufficient to know that a given setting has a "default" value or a non-default value.

It might also be helpful if you can enumerate the possible combinations of values and the corresponding source/explicit value that they mean.

- Sudheer

> On Nov 21, 2016, at 12:52 PM, Alan Carroll <so...@yahoo-inc.com.INVALID> wrote:
> 
> No, they behave very differently. If you have a config variable in RecordsConfig.cc and it's not mentioned in records.config you get REC_SOURCE_DEFAULT. If you have a plugin created config variable that is not in records.config you get REC_SOURCE_EXPLICIT. In both cases if the value is in records.config you get REC_SOURCE_EXPLICIT. So for a built in it is detectable if the value is in records.config, but not detectable for a plugin created variable. 
> 
>    On Monday, November 21, 2016 2:34 PM, James Peach <jp...@apache.org> wrote:
> 
> 
> 
>> On Nov 21, 2016, at 11:50 AM, Alan Carroll <so...@yahoo-inc.com.INVALID> wrote:
>> 
>> No, the source is REC_SOURCE_EXPLICIT even if the variable is not mentioned in records.config. How else can a plugin tell if a config variable it creates has a value in records.config?
> 
> I think I’m confused about what you are proposing then. It sounds like configuration records used by plugins behave just like configuration records used by traffic_server. Is that the case? If so, it sounds like correct behavior to me. If not, what is the problem that you are addressing here?
> 
>>   
>> 
>>     On Monday, November 21, 2016 12:21 PM, James Peach <jp...@apache.org> wrote:
>> 
>> 
>> 
>>> On Nov 21, 2016, at 9:14 AM, Alan Carroll <so...@yahoo-inc.com.INVALID> wrote:
>>> 
>>> REC_SOURCE_EXPLICIT - see TSMgmtStringCreate in InkAPI.cc (around line 8896). That is the same value as setting the value in a configuration file.
>> 
>> That sounds correct to me. The value was set in a configuration file and is being consumed by a plugin.
>> 
>>>     On Monday, November 21, 2016 10:29 AM, James Peach <jp...@apache.org> wrote:
>>> 
>>> 
>>> 
>>>> On Nov 21, 2016, at 7:20 AM, Alan Carroll <so...@yahoo-inc.com.INVALID> wrote:
>>>> 
>>>> The config variable sourcing data was originally put in to be able to detect that a variable was set even if that setting was the same as the default (otherwise this is impossible to reliably determine). I am working with a plugin that defines its own config records but as far as I can tell (1) the source value for a plugin configured record default is the same as for an explicitly set value and (2) the source information is not available via the plugin API.
>>> 
>>> What is the source when you specify a record in records.config, then register it from a plugin?
>>> 
>>>> I would like to fix both of these.
>>>> First, add a "PLUGIN" value to RecSourceT to indicate the value is set as a default by the plugin and use EXPLICIT only for values from configuration files or explicit value setting API calls.
>>>> Second,
>>>> * Move or copy the RecSourceT define to apidef.h.in.* Add TSMgmtSourceGet(const char*) to return the RecSourceT value for a configuration variable.
>>>> 
>>> 
>>> 
>> 
>> 
> 
>

Re: Config variable source values

Posted by Alan Carroll <so...@yahoo-inc.com.INVALID>.

No, they behave very differently. If you have a config variable in RecordsConfig.cc and it's not mentioned in records.config you get REC_SOURCE_DEFAULT. If you have a plugin created config variable that is not in records.config you get REC_SOURCE_EXPLICIT. In both cases if the value is in records.config you get REC_SOURCE_EXPLICIT. So for a built in it is detectable if the value is in records.config, but not detectable for a plugin created variable. 

    On Monday, November 21, 2016 2:34 PM, James Peach <jp...@apache.org> wrote:
 

 
> On Nov 21, 2016, at 11:50 AM, Alan Carroll <so...@yahoo-inc.com.INVALID> wrote:
> 
> No, the source is REC_SOURCE_EXPLICIT even if the variable is not mentioned in records.config. How else can a plugin tell if a config variable it creates has a value in records.config?

I think I’m confused about what you are proposing then. It sounds like configuration records used by plugins behave just like configuration records used by traffic_server. Is that the case? If so, it sounds like correct behavior to me. If not, what is the problem that you are addressing here?

>  
> 
>    On Monday, November 21, 2016 12:21 PM, James Peach <jp...@apache.org> wrote:
> 
> 
> 
>> On Nov 21, 2016, at 9:14 AM, Alan Carroll <so...@yahoo-inc.com.INVALID> wrote:
>> 
>> REC_SOURCE_EXPLICIT - see TSMgmtStringCreate in InkAPI.cc (around line 8896). That is the same value as setting the value in a configuration file.
> 
> That sounds correct to me. The value was set in a configuration file and is being consumed by a plugin.
> 
>>    On Monday, November 21, 2016 10:29 AM, James Peach <jp...@apache.org> wrote:
>> 
>> 
>> 
>>> On Nov 21, 2016, at 7:20 AM, Alan Carroll <so...@yahoo-inc.com.INVALID> wrote:
>>> 
>>> The config variable sourcing data was originally put in to be able to detect that a variable was set even if that setting was the same as the default (otherwise this is impossible to reliably determine). I am working with a plugin that defines its own config records but as far as I can tell (1) the source value for a plugin configured record default is the same as for an explicitly set value and (2) the source information is not available via the plugin API.
>> 
>> What is the source when you specify a record in records.config, then register it from a plugin?
>> 
>>> I would like to fix both of these.
>>> First, add a "PLUGIN" value to RecSourceT to indicate the value is set as a default by the plugin and use EXPLICIT only for values from configuration files or explicit value setting API calls.
>>> Second,
>>> * Move or copy the RecSourceT define to apidef.h.in.* Add TSMgmtSourceGet(const char*) to return the RecSourceT value for a configuration variable.
>>> 
>> 
>> 
> 
>

Re: Config variable source values

Posted by James Peach <jp...@apache.org>.

> On Nov 21, 2016, at 11:50 AM, Alan Carroll <so...@yahoo-inc.com.INVALID> wrote:
> 
> No, the source is REC_SOURCE_EXPLICIT even if the variable is not mentioned in records.config. How else can a plugin tell if a config variable it creates has a value in records.config?

I think I’m confused about what you are proposing then. It sounds like configuration records used by plugins behave just like configuration records used by traffic_server. Is that the case? If so, it sounds like correct behavior to me. If not, what is the problem that you are addressing here?

>  
> 
>    On Monday, November 21, 2016 12:21 PM, James Peach <jp...@apache.org> wrote:
> 
> 
> 
>> On Nov 21, 2016, at 9:14 AM, Alan Carroll <so...@yahoo-inc.com.INVALID> wrote:
>> 
>> REC_SOURCE_EXPLICIT - see TSMgmtStringCreate in InkAPI.cc (around line 8896). That is the same value as setting the value in a configuration file.
> 
> That sounds correct to me. The value was set in a configuration file and is being consumed by a plugin.
> 
>>     On Monday, November 21, 2016 10:29 AM, James Peach <jp...@apache.org> wrote:
>> 
>> 
>> 
>>> On Nov 21, 2016, at 7:20 AM, Alan Carroll <so...@yahoo-inc.com.INVALID> wrote:
>>> 
>>> The config variable sourcing data was originally put in to be able to detect that a variable was set even if that setting was the same as the default (otherwise this is impossible to reliably determine). I am working with a plugin that defines its own config records but as far as I can tell (1) the source value for a plugin configured record default is the same as for an explicitly set value and (2) the source information is not available via the plugin API.
>> 
>> What is the source when you specify a record in records.config, then register it from a plugin?
>> 
>>> I would like to fix both of these.
>>> First, add a "PLUGIN" value to RecSourceT to indicate the value is set as a default by the plugin and use EXPLICIT only for values from configuration files or explicit value setting API calls.
>>> Second,
>>> * Move or copy the RecSourceT define to apidef.h.in.* Add TSMgmtSourceGet(const char*) to return the RecSourceT value for a configuration variable.
>>> 
>> 
>> 
> 
>

Re: Config variable source values

Posted by Alan Carroll <so...@yahoo-inc.com.INVALID>.

No, the source is REC_SOURCE_EXPLICIT even if the variable is not mentioned in records.config. How else can a plugin tell if a config variable it creates has a value in records.config? 

    On Monday, November 21, 2016 12:21 PM, James Peach <jp...@apache.org> wrote:
 

 
> On Nov 21, 2016, at 9:14 AM, Alan Carroll <so...@yahoo-inc.com.INVALID> wrote:
> 
> REC_SOURCE_EXPLICIT - see TSMgmtStringCreate in InkAPI.cc (around line 8896). That is the same value as setting the value in a configuration file.

That sounds correct to me. The value was set in a configuration file and is being consumed by a plugin.

>    On Monday, November 21, 2016 10:29 AM, James Peach <jp...@apache.org> wrote:
> 
> 
> 
>> On Nov 21, 2016, at 7:20 AM, Alan Carroll <so...@yahoo-inc.com.INVALID> wrote:
>> 
>> The config variable sourcing data was originally put in to be able to detect that a variable was set even if that setting was the same as the default (otherwise this is impossible to reliably determine). I am working with a plugin that defines its own config records but as far as I can tell (1) the source value for a plugin configured record default is the same as for an explicitly set value and (2) the source information is not available via the plugin API.
> 
> What is the source when you specify a record in records.config, then register it from a plugin?
> 
>> I would like to fix both of these.
>> First, add a "PLUGIN" value to RecSourceT to indicate the value is set as a default by the plugin and use EXPLICIT only for values from configuration files or explicit value setting API calls.
>> Second,
>> * Move or copy the RecSourceT define to apidef.h.in.* Add TSMgmtSourceGet(const char*) to return the RecSourceT value for a configuration variable.
>> 
> 
>

Re: Config variable source values

Posted by James Peach <jp...@apache.org>.

> On Nov 21, 2016, at 9:14 AM, Alan Carroll <so...@yahoo-inc.com.INVALID> wrote:
> 
> REC_SOURCE_EXPLICIT - see TSMgmtStringCreate in InkAPI.cc (around line 8896). That is the same value as setting the value in a configuration file.

That sounds correct to me. The value was set in a configuration file and is being consumed by a plugin.

>    On Monday, November 21, 2016 10:29 AM, James Peach <jp...@apache.org> wrote:
> 
> 
> 
>> On Nov 21, 2016, at 7:20 AM, Alan Carroll <so...@yahoo-inc.com.INVALID> wrote:
>> 
>> The config variable sourcing data was originally put in to be able to detect that a variable was set even if that setting was the same as the default (otherwise this is impossible to reliably determine). I am working with a plugin that defines its own config records but as far as I can tell (1) the source value for a plugin configured record default is the same as for an explicitly set value and (2) the source information is not available via the plugin API.
> 
> What is the source when you specify a record in records.config, then register it from a plugin?
> 
>> I would like to fix both of these.
>> First, add a "PLUGIN" value to RecSourceT to indicate the value is set as a default by the plugin and use EXPLICIT only for values from configuration files or explicit value setting API calls.
>> Second,
>> * Move or copy the RecSourceT define to apidef.h.in.* Add TSMgmtSourceGet(const char*) to return the RecSourceT value for a configuration variable.
>> 
> 
>

Re: Config variable source values

Posted by Alan Carroll <so...@yahoo-inc.com.INVALID>.

REC_SOURCE_EXPLICIT - see TSMgmtStringCreate in InkAPI.cc (around line 8896). That is the same value as setting the value in a configuration file. 

    On Monday, November 21, 2016 10:29 AM, James Peach <jp...@apache.org> wrote:
 

 
> On Nov 21, 2016, at 7:20 AM, Alan Carroll <so...@yahoo-inc.com.INVALID> wrote:
> 
> The config variable sourcing data was originally put in to be able to detect that a variable was set even if that setting was the same as the default (otherwise this is impossible to reliably determine). I am working with a plugin that defines its own config records but as far as I can tell (1) the source value for a plugin configured record default is the same as for an explicitly set value and (2) the source information is not available via the plugin API.

What is the source when you specify a record in records.config, then register it from a plugin?

> I would like to fix both of these.
> First, add a "PLUGIN" value to RecSourceT to indicate the value is set as a default by the plugin and use EXPLICIT only for values from configuration files or explicit value setting API calls.
> Second,
> * Move or copy the RecSourceT define to apidef.h.in.* Add TSMgmtSourceGet(const char*) to return the RecSourceT value for a configuration variable.
>

Re: Config variable source values

Posted by James Peach <jp...@apache.org>.

> On Nov 21, 2016, at 7:20 AM, Alan Carroll <so...@yahoo-inc.com.INVALID> wrote:
> 
> The config variable sourcing data was originally put in to be able to detect that a variable was set even if that setting was the same as the default (otherwise this is impossible to reliably determine). I am working with a plugin that defines its own config records but as far as I can tell (1) the source value for a plugin configured record default is the same as for an explicitly set value and (2) the source information is not available via the plugin API.

What is the source when you specify a record in records.config, then register it from a plugin?

> I would like to fix both of these.
> First, add a "PLUGIN" value to RecSourceT to indicate the value is set as a default by the plugin and use EXPLICIT only for values from configuration files or explicit value setting API calls.
> Second,
> * Move or copy the RecSourceT define to apidef.h.in.* Add TSMgmtSourceGet(const char*) to return the RecSourceT value for a configuration variable.
>