question on [STACK]/[SERVICE]/metainfo.xml inheritance rules

[Alexander Denissov <ad...@pivotal.io>]

I am trying to understand the inheritance rules that govern services
metainfo.xml file contents. I looked at
https://issues.apache.org/jira/browse/AMBARI-2819 but it didn't answer the
following:

1. If a component exists in the parent stack and is defined again in the
child stack with just a few attributes, are these values just to override
the parent's values or the whole component definition is replaced.

Example: HDP-2.1 YARN/metainfo.xml contains definition of RESOURCEMANAGER
with just 4 attributes, out of which only the value for "cardinality" is
different one in HDP-2.0.6 definition. But 2.0.6 definition also has a lot
more attributes (such as custom commands) that are not mentioned in 2.1.
Will these "missing" attributes be inherited by 2.1 stack ? If yes, why
other attributes (category and configuration-dependencies) are defined
again with the same values instead of being inherited ?

2. If a component is missing in the new definition but is present in the
parent, does it get inherited ?

3. Configuration dependencies for the service -- are they overwritten or
merged ?

Example: HDP-2.1 YARN/metainfo.xml contains <configuration-dependencies>
element with 4 <config-type>, where as in HDP-2.0.6 the same element has
5 <config-type> (extra line is mapred-site). So will <config-type>mapred
-site</config-type> be inherited and present in 2.1 definition or was this
the way to get rid of this specific line for the new stack ?

4. What about other elements in metainfo.xml -- which rules apply ?

--
Thanks,
Alex.

Re: question on [STACK]/[SERVICE]/metainfo.xml inheritance rules

[Jeff Sposetti <je...@hortonworks.com>]

Captured and started this wiki page:

https://cwiki.apache.org/confluence/display/AMBARI/FAQ#

On Fri, Sep 5, 2014 at 9:54 AM, Sumit Mohanty <sm...@hortonworks.com> wrote:
> Could we save these as FAQs on the Ambari wiki?
>
> -Sumit
>
>
> On Thu, Sep 4, 2014 at 5:53 PM, Siddharth Wagle <sw...@hortonworks.com>
> wrote:
>>
>> Hi Alex,
>>
>> Replies inline.
>>
>> 1. If a component exists in the parent stack and is defined again in the
>> child stack with just a few attributes, are these values just to override
>> the parent's values or the whole component definition is replaced.
>>
>> We go property by property and merge them from parent to child. So if you
>> remove a category for example from the child it will be inherited from
>> parent, that goes for pretty much all properties.
>> So, the question is how do we tackle existence of a property in both
>> parent and child. Here, most of the decision still follow same paradigm as
>> take the child value instead of parent and every property in parent, not
>> explicitly deleted from child using a marker like <deleted> tag, is included
>> in the merge.
>>
>> - For config-dependencies, we take all or nothing approach, if this
>> property exists in child use it and all of its children else take it from
>> parent.
>> - The custom commands are merged based on names, such that merged
>> definition is a union of commands with child commands with same name
>> overriding those fro parent.
>> - Cardinality is overwritten by a child or take from the parent if child
>> has not provided one.
>>
>> You could look at this method for more details:
>> org.apache.ambari.server.api.util.StackExtensionHelper#mergeServices
>>
>> 2. If a component is missing in the new definition but is present in the
>> parent, does it get inherited ?
>>
>> Generally yes.
>>
>> 3. Configuration dependencies for the service -- are they overwritten or
>> merged ?
>>
>> Overwritten.
>>
>> 4. What about other elements in metainfo.xml -- which rules apply ?
>>
>> Answered in 1.
>>
>> -Sid
>>
>>
>>
>>
>>
>>
>> On Thu, Sep 4, 2014 at 5:02 PM, Alexander Denissov <ad...@pivotal.io>
>> wrote:
>>>
>>> I am trying to understand the inheritance rules that govern services
>>> metainfo.xml file contents. I looked at
>>> https://issues.apache.org/jira/browse/AMBARI-2819 but it didn't answer the
>>> following:
>>>
>>> 1. If a component exists in the parent stack and is defined again in the
>>> child stack with just a few attributes, are these values just to override
>>> the parent's values or the whole component definition is replaced.
>>>
>>> Example: HDP-2.1 YARN/metainfo.xml contains definition of RESOURCEMANAGER
>>> with just 4 attributes, out of which only the value for "cardinality" is
>>> different one in HDP-2.0.6 definition. But 2.0.6 definition also has a lot
>>> more attributes (such as custom commands) that are not mentioned in 2.1.
>>> Will these "missing" attributes be inherited by 2.1 stack ? If yes, why
>>> other attributes (category and configuration-dependencies) are defined again
>>> with the same values instead of being inherited ?
>>>
>>> 2. If a component is missing in the new definition but is present in the
>>> parent, does it get inherited ?
>>>
>>> 3. Configuration dependencies for the service -- are they overwritten or
>>> merged ?
>>>
>>> Example: HDP-2.1 YARN/metainfo.xml contains <configuration-dependencies>
>>> element with 4 <config-type>, where as in HDP-2.0.6 the same element has 5
>>> <config-type> (extra line is mapred-site). So will
>>> <config-type>mapred-site</config-type> be inherited and present in 2.1
>>> definition or was this the way to get rid of this specific line for the new
>>> stack ?
>>>
>>> 4. What about other elements in metainfo.xml -- which rules apply ?
>>>
>>> --
>>> Thanks,
>>> Alex.
>>
>>
>>
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader of
>> this message is not the intended recipient, you are hereby notified that any
>> printing, copying, dissemination, distribution, disclosure or forwarding of
>> this communication is strictly prohibited. If you have received this
>> communication in error, please contact the sender immediately and delete it
>> from your system. Thank You.
>
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader of
> this message is not the intended recipient, you are hereby notified that any
> printing, copying, dissemination, distribution, disclosure or forwarding of
> this communication is strictly prohibited. If you have received this
> communication in error, please contact the sender immediately and delete it
> from your system. Thank You.

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: question on [STACK]/[SERVICE]/metainfo.xml inheritance rules

[Sumit Mohanty <sm...@hortonworks.com>]

Could we save these as FAQs on the Ambari wiki?

-Sumit


On Thu, Sep 4, 2014 at 5:53 PM, Siddharth Wagle <sw...@hortonworks.com>
wrote:

> Hi Alex,
>
> Replies inline.
>
> 1. If a component exists in the parent stack and is defined again in the
> child stack with just a few attributes, are these values just to override
> the parent's values or the whole component definition is replaced.
>
> We go property by property and merge them from parent to child. So if you
> remove a category for example from the child it will be inherited from
> parent, that goes for pretty much all properties.
> So, the question is how do we tackle existence of a property in both
> parent and child. Here, most of the decision still follow same paradigm as
> take the child value instead of parent and every property in parent, not
> explicitly deleted from child using a marker like <deleted> tag, is
> included in the merge.
>
> - For config-dependencies, we take all or nothing approach, if this
> property exists in child use it and all of its children else take it from
> parent.
> - The custom commands are merged based on names, such that merged
> definition is a union of commands with child commands with same name
> overriding those fro parent.
> - Cardinality is overwritten by a child or take from the parent if child
> has not provided one.
>
> You could look at this method for more details:
> org.apache.ambari.server.api.util.StackExtensionHelper#mergeServices
>
> 2. If a component is missing in the new definition but is present in the
> parent, does it get inherited ?
>
> Generally yes.
>
> 3. Configuration dependencies for the service -- are they overwritten or
> merged ?
>
> Overwritten.
>
> 4. What about other elements in metainfo.xml -- which rules apply ?
>
> Answered in 1.
>
> -Sid
>
>
>
>
>
>
> On Thu, Sep 4, 2014 at 5:02 PM, Alexander Denissov <ad...@pivotal.io>
> wrote:
>
>> I am trying to understand the inheritance rules that govern services
>> metainfo.xml file contents. I looked at
>> https://issues.apache.org/jira/browse/AMBARI-2819 but it didn't answer
>> the following:
>>
>> 1. If a component exists in the parent stack and is defined again in the
>> child stack with just a few attributes, are these values just to override
>> the parent's values or the whole component definition is replaced.
>>
>> Example: HDP-2.1 YARN/metainfo.xml contains definition of RESOURCEMANAGER
>> with just 4 attributes, out of which only the value for "cardinality" is
>> different one in HDP-2.0.6 definition. But 2.0.6 definition also has a lot
>> more attributes (such as custom commands) that are not mentioned in 2.1.
>> Will these "missing" attributes be inherited by 2.1 stack ? If yes, why
>> other attributes (category and configuration-dependencies) are defined
>> again with the same values instead of being inherited ?
>>
>> 2. If a component is missing in the new definition but is present in the
>> parent, does it get inherited ?
>>
>> 3. Configuration dependencies for the service -- are they overwritten or
>> merged ?
>>
>> Example: HDP-2.1 YARN/metainfo.xml contains <configuration-dependencies>
>> element with 4 <config-type>, where as in HDP-2.0.6 the same element has
>> 5 <config-type> (extra line is mapred-site). So will <config-type>mapred
>> -site</config-type> be inherited and present in 2.1 definition or was
>> this the way to get rid of this specific line for the new stack ?
>>
>> 4. What about other elements in metainfo.xml -- which rules apply ?
>>
>> --
>> Thanks,
>> Alex.
>>
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: question on [STACK]/[SERVICE]/metainfo.xml inheritance rules

[Siddharth Wagle <sw...@hortonworks.com>]

Hi Alex,

Replies inline.

1. If a component exists in the parent stack and is defined again in the
child stack with just a few attributes, are these values just to override
the parent's values or the whole component definition is replaced.

We go property by property and merge them from parent to child. So if you
remove a category for example from the child it will be inherited from
parent, that goes for pretty much all properties.
So, the question is how do we tackle existence of a property in both parent
and child. Here, most of the decision still follow same paradigm as take
the child value instead of parent and every property in parent, not
explicitly deleted from child using a marker like <deleted> tag, is
included in the merge.

- For config-dependencies, we take all or nothing approach, if this
property exists in child use it and all of its children else take it from
parent.
- The custom commands are merged based on names, such that merged
definition is a union of commands with child commands with same name
overriding those fro parent.
- Cardinality is overwritten by a child or take from the parent if child
has not provided one.

You could look at this method for more details:
org.apache.ambari.server.api.util.StackExtensionHelper#mergeServices

2. If a component is missing in the new definition but is present in the
parent, does it get inherited ?

Generally yes.

3. Configuration dependencies for the service -- are they overwritten or
merged ?

Overwritten.

4. What about other elements in metainfo.xml -- which rules apply ?

Answered in 1.

-Sid






On Thu, Sep 4, 2014 at 5:02 PM, Alexander Denissov <ad...@pivotal.io>
wrote:

> I am trying to understand the inheritance rules that govern services
> metainfo.xml file contents. I looked at
> https://issues.apache.org/jira/browse/AMBARI-2819 but it didn't answer
> the following:
>
> 1. If a component exists in the parent stack and is defined again in the
> child stack with just a few attributes, are these values just to override
> the parent's values or the whole component definition is replaced.
>
> Example: HDP-2.1 YARN/metainfo.xml contains definition of RESOURCEMANAGER
> with just 4 attributes, out of which only the value for "cardinality" is
> different one in HDP-2.0.6 definition. But 2.0.6 definition also has a lot
> more attributes (such as custom commands) that are not mentioned in 2.1.
> Will these "missing" attributes be inherited by 2.1 stack ? If yes, why
> other attributes (category and configuration-dependencies) are defined
> again with the same values instead of being inherited ?
>
> 2. If a component is missing in the new definition but is present in the
> parent, does it get inherited ?
>
> 3. Configuration dependencies for the service -- are they overwritten or
> merged ?
>
> Example: HDP-2.1 YARN/metainfo.xml contains <configuration-dependencies>
> element with 4 <config-type>, where as in HDP-2.0.6 the same element has
> 5 <config-type> (extra line is mapred-site). So will <config-type>mapred
> -site</config-type> be inherited and present in 2.1 definition or was
> this the way to get rid of this specific line for the new stack ?
>
> 4. What about other elements in metainfo.xml -- which rules apply ?
>
> --
> Thanks,
> Alex.
>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.