You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ambari.apache.org by Lukáš Drbal <lu...@gmail.com> on 2017/01/09 16:17:32 UTC
Ambari 2.4 - custom service and condition package instalation
Hi,
i have custom service which provides more components [1].
All components are installed from deb packages and i don't wanna install
all packages on all nodes (just component specific) so i try to use
condition [2] but it ends with error:
"resource_management.core.exceptions.Fail: Condition with name
'should_install_confluent_supervisor', when installing package supervisor.
Please check package_conditions.py."
I check source code and it look's [3] like there is hardcoded package
"resource_management.libraries.functions".
In 2.2 i was able to use param exclude_packages in install method. How i
can achieve same functionality in 2.4? Did i miss something?
Thanks for help.
[1]
I have list of all packages in metainfo.xml
For example:
<osSpecifics>
<osSpecific>
<osFamily>ubuntu14</osFamily>
<packages>
<package>
<name>supervisor</name>
<condition>should_install_supervisor</condition>
</package>
<package>
<name>apache</name>
<condition>should_install_apache</condition>
</package>
</packages>
</<osSpecific>
</osSpecifics>
def should_install_supervisor():
config = Script.get_config()
# _has_applicable_local_component is copied
from ambari-common/src/main/python/resource_management/libraries/functions/package_conditions.py
return _has_applicable_local_component(config, ["MY_SERVICE"])
[3]
https://github.com/apache/ambari/blob/release-2.4.2/ambari-common/src/main/python/resource_management/libraries/script/script.py#L592
--
Save The World - http://www.worldcommunitygrid.org/
http://www.worldcommunitygrid.org/stat/viewMemberInfo.do?userName=LesTR
LesTR
Re: Ambari 2.4 - custom service and condition package instalation
Posted by Lukáš Drbal <lu...@gmail.com>.
created AMBARI-19653
On Tue, Jan 17, 2017 at 9:06 PM, Lukáš Drbal <lu...@gmail.com> wrote:
> Hmm, my workaround works until i try upgrade HDP stack. There is another
> hard coded package. I'll investigate it more Tomorrow but now it looks like
> big race condition - I am not able to specify deb package per component
> without changing ambari server and agent code. This was more clear in 2.2
> version.
>
> Am i only one who need more control on installed package in custom
> service?
>
> On Thu, Jan 12, 2017 at 11:30 AM, Lukáš Drbal <lu...@gmail.com>
> wrote:
>
>> Hi,
>>
>> i found an workaround - copy method install_packages into my class and
>> replace that hardcoded Script [1]
>>
>> Its different in trunk [2] but package resource_management.libraries.functions
>> is still hardcoded.
>>
>> I'll create an Jira for that because it breaks functionality and we don't
>> want change ambari source code.
>>
>> ----
>> [1] https://github.com/apache/ambari/blob/release-2.4.2/amba
>> ri-common/src/main/python/resource_management/libraries/scri
>> pt/script.py#L556
>> [2] https://github.com/apache/ambari/blob/trunk/ambari-commo
>> n/src/main/python/resource_management/libraries/script/script.py#L625
>>
>> On Mon, Jan 9, 2017 at 9:19 PM, Lukáš Drbal <lu...@gmail.com>
>> wrote:
>>
>>> Hi Alejandro,
>>>
>>> thank you for response! I definitely don't, because it means change
>>> ambari source code just because adding new service and this wasn't needed
>>> in 2.2. Am i correct?
>>> If yes, that's strange :-(
>>>
>>> Lukas
>>>
>>> On Mon, Jan 9, 2017 at 8:16 PM, Alejandro Fernandez <
>>> afernandez@hortonworks.com> wrote:
>>>
>>>> Hi Lukas,
>>>>
>>>> It's not finding the method names "should_install_supervisor", "
>>>> should_install_apache"
>>>>
>>>> Did you add them to this list?
>>>>
>>>> __all__ = ["is_lzo_enabled", "should_install_phoenix", "should_install_ams_collector", "should_install_ams_grafana",
>>>> "should_install_mysql", "should_install_mysql_connector", "should_install_ranger_tagsync"]
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> Alejandro
>>>>
>>>> From: Lukáš Drbal <lu...@gmail.com>
>>>> Reply-To: "user@ambari.apache.org" <us...@ambari.apache.org>
>>>> Date: Monday, January 9, 2017 at 8:17 AM
>>>> To: "user@ambari.apache.org" <us...@ambari.apache.org>
>>>> Subject: Ambari 2.4 - custom service and condition package instalation
>>>>
>>>> Hi,
>>>>
>>>> i have custom service which provides more components [1].
>>>> All components are installed from deb packages and i don't wanna
>>>> install all packages on all nodes (just component specific) so i try to use
>>>> condition [2] but it ends with error:
>>>> "resource_management.core.exceptions.Fail: Condition with name
>>>> 'should_install_confluent_supervisor', when installing package
>>>> supervisor. Please check package_conditions.py."
>>>>
>>>> I check source code and it look's [3] like there is hardcoded package
>>>> "resource_management.libraries.functions".
>>>>
>>>> In 2.2 i was able to use param exclude_packages in install method. How
>>>> i can achieve same functionality in 2.4? Did i miss something?
>>>>
>>>> Thanks for help.
>>>>
>>>> [1]
>>>> I have list of all packages in metainfo.xml
>>>> For example:
>>>> <osSpecifics>
>>>> <osSpecific>
>>>> <osFamily>ubuntu14</osFamily>
>>>> <packages>
>>>> <package>
>>>> <name>supervisor</name>
>>>> <condition>should_install_supervisor</condition>
>>>> </package>
>>>> <package>
>>>> <name>apache</name>
>>>> <condition>should_install_apache</condition>
>>>> </package>
>>>> </packages>
>>>> </<osSpecific>
>>>> </osSpecifics>
>>>>
>>>> def should_install_supervisor():
>>>> config = Script.get_config()
>>>> # _has_applicable_local_component is copied
>>>> from ambari-common/src/main/python/resource_management/libra
>>>> ries/functions/package_conditions.py
>>>> return _has_applicable_local_component(config, ["MY_SERVICE"])
>>>>
>>>> [3] https://github.com/apache/ambari/blob/release-2.4.2/ambari-
>>>> common/src/main/python/resource_management/libraries/script/
>>>> script.py#L592
>>>>
>>>> --
>>>> Save The World - http://www.worldcommunitygrid.org/
>>>> http://www.worldcommunitygrid.org/stat/viewMemberInfo.do?userName=LesTR
>>>>
>>>> LesTR
>>>>
>>>
>>>
>>>
>>> --
>>> Save The World - http://www.worldcommunitygrid.org/
>>> http://www.worldcommunitygrid.org/stat/viewMemberInfo.do?userName=LesTR
>>>
>>> LesTR
>>>
>>
>>
>>
>> --
>> Save The World - http://www.worldcommunitygrid.org/
>> http://www.worldcommunitygrid.org/stat/viewMemberInfo.do?userName=LesTR
>>
>> LesTR
>>
>
>
>
> --
> Save The World - http://www.worldcommunitygrid.org/
> http://www.worldcommunitygrid.org/stat/viewMemberInfo.do?userName=LesTR
>
> LesTR
>
--
Save The World - http://www.worldcommunitygrid.org/
http://www.worldcommunitygrid.org/stat/viewMemberInfo.do?userName=LesTR
LesTR
Re: Ambari 2.4 - custom service and condition package instalation
Posted by Lukáš Drbal <lu...@gmail.com>.
Hmm, my workaround works until i try upgrade HDP stack. There is another
hard coded package. I'll investigate it more Tomorrow but now it looks like
big race condition - I am not able to specify deb package per component
without changing ambari server and agent code. This was more clear in 2.2
version.
Am i only one who need more control on installed package in custom service?
On Thu, Jan 12, 2017 at 11:30 AM, Lukáš Drbal <lu...@gmail.com> wrote:
> Hi,
>
> i found an workaround - copy method install_packages into my class and
> replace that hardcoded Script [1]
>
> Its different in trunk [2] but package resource_management.libraries.functions
> is still hardcoded.
>
> I'll create an Jira for that because it breaks functionality and we don't
> want change ambari source code.
>
> ----
> [1] https://github.com/apache/ambari/blob/release-2.4.2/
> ambari-common/src/main/python/resource_management/libraries/
> script/script.py#L556
> [2] https://github.com/apache/ambari/blob/trunk/ambari-
> common/src/main/python/resource_management/libraries/script/script.py#L625
>
> On Mon, Jan 9, 2017 at 9:19 PM, Lukáš Drbal <lu...@gmail.com> wrote:
>
>> Hi Alejandro,
>>
>> thank you for response! I definitely don't, because it means change
>> ambari source code just because adding new service and this wasn't needed
>> in 2.2. Am i correct?
>> If yes, that's strange :-(
>>
>> Lukas
>>
>> On Mon, Jan 9, 2017 at 8:16 PM, Alejandro Fernandez <
>> afernandez@hortonworks.com> wrote:
>>
>>> Hi Lukas,
>>>
>>> It's not finding the method names "should_install_supervisor", "
>>> should_install_apache"
>>>
>>> Did you add them to this list?
>>>
>>> __all__ = ["is_lzo_enabled", "should_install_phoenix", "should_install_ams_collector", "should_install_ams_grafana",
>>> "should_install_mysql", "should_install_mysql_connector", "should_install_ranger_tagsync"]
>>>
>>>
>>> Thanks,
>>>
>>> Alejandro
>>>
>>> From: Lukáš Drbal <lu...@gmail.com>
>>> Reply-To: "user@ambari.apache.org" <us...@ambari.apache.org>
>>> Date: Monday, January 9, 2017 at 8:17 AM
>>> To: "user@ambari.apache.org" <us...@ambari.apache.org>
>>> Subject: Ambari 2.4 - custom service and condition package instalation
>>>
>>> Hi,
>>>
>>> i have custom service which provides more components [1].
>>> All components are installed from deb packages and i don't wanna install
>>> all packages on all nodes (just component specific) so i try to use
>>> condition [2] but it ends with error:
>>> "resource_management.core.exceptions.Fail: Condition with name
>>> 'should_install_confluent_supervisor', when installing package
>>> supervisor. Please check package_conditions.py."
>>>
>>> I check source code and it look's [3] like there is hardcoded package
>>> "resource_management.libraries.functions".
>>>
>>> In 2.2 i was able to use param exclude_packages in install method. How i
>>> can achieve same functionality in 2.4? Did i miss something?
>>>
>>> Thanks for help.
>>>
>>> [1]
>>> I have list of all packages in metainfo.xml
>>> For example:
>>> <osSpecifics>
>>> <osSpecific>
>>> <osFamily>ubuntu14</osFamily>
>>> <packages>
>>> <package>
>>> <name>supervisor</name>
>>> <condition>should_install_supervisor</condition>
>>> </package>
>>> <package>
>>> <name>apache</name>
>>> <condition>should_install_apache</condition>
>>> </package>
>>> </packages>
>>> </<osSpecific>
>>> </osSpecifics>
>>>
>>> def should_install_supervisor():
>>> config = Script.get_config()
>>> # _has_applicable_local_component is copied
>>> from ambari-common/src/main/python/resource_management/libra
>>> ries/functions/package_conditions.py
>>> return _has_applicable_local_component(config, ["MY_SERVICE"])
>>>
>>> [3] https://github.com/apache/ambari/blob/release-2.4.2/ambari-
>>> common/src/main/python/resource_management/libraries/script/
>>> script.py#L592
>>>
>>> --
>>> Save The World - http://www.worldcommunitygrid.org/
>>> http://www.worldcommunitygrid.org/stat/viewMemberInfo.do?userName=LesTR
>>>
>>> LesTR
>>>
>>
>>
>>
>> --
>> Save The World - http://www.worldcommunitygrid.org/
>> http://www.worldcommunitygrid.org/stat/viewMemberInfo.do?userName=LesTR
>>
>> LesTR
>>
>
>
>
> --
> Save The World - http://www.worldcommunitygrid.org/
> http://www.worldcommunitygrid.org/stat/viewMemberInfo.do?userName=LesTR
>
> LesTR
>
--
Save The World - http://www.worldcommunitygrid.org/
http://www.worldcommunitygrid.org/stat/viewMemberInfo.do?userName=LesTR
LesTR
Re: Ambari 2.4 - custom service and condition package instalation
Posted by Lukáš Drbal <lu...@gmail.com>.
Hi,
i found an workaround - copy method install_packages into my class and
replace that hardcoded Script [1]
Its different in trunk [2] but
package resource_management.libraries.functions is still hardcoded.
I'll create an Jira for that because it breaks functionality and we don't
want change ambari source code.
----
[1]
https://github.com/apache/ambari/blob/release-2.4.2/ambari-common/src/main/python/resource_management/libraries/script/script.py#L556
[2]
https://github.com/apache/ambari/blob/trunk/ambari-common/src/main/python/resource_management/libraries/script/script.py#L625
On Mon, Jan 9, 2017 at 9:19 PM, Lukáš Drbal <lu...@gmail.com> wrote:
> Hi Alejandro,
>
> thank you for response! I definitely don't, because it means change
> ambari source code just because adding new service and this wasn't needed
> in 2.2. Am i correct?
> If yes, that's strange :-(
>
> Lukas
>
> On Mon, Jan 9, 2017 at 8:16 PM, Alejandro Fernandez <
> afernandez@hortonworks.com> wrote:
>
>> Hi Lukas,
>>
>> It's not finding the method names "should_install_supervisor", "
>> should_install_apache"
>>
>> Did you add them to this list?
>>
>> __all__ = ["is_lzo_enabled", "should_install_phoenix", "should_install_ams_collector", "should_install_ams_grafana",
>> "should_install_mysql", "should_install_mysql_connector", "should_install_ranger_tagsync"]
>>
>>
>> Thanks,
>>
>> Alejandro
>>
>> From: Lukáš Drbal <lu...@gmail.com>
>> Reply-To: "user@ambari.apache.org" <us...@ambari.apache.org>
>> Date: Monday, January 9, 2017 at 8:17 AM
>> To: "user@ambari.apache.org" <us...@ambari.apache.org>
>> Subject: Ambari 2.4 - custom service and condition package instalation
>>
>> Hi,
>>
>> i have custom service which provides more components [1].
>> All components are installed from deb packages and i don't wanna install
>> all packages on all nodes (just component specific) so i try to use
>> condition [2] but it ends with error:
>> "resource_management.core.exceptions.Fail: Condition with name
>> 'should_install_confluent_supervisor', when installing package
>> supervisor. Please check package_conditions.py."
>>
>> I check source code and it look's [3] like there is hardcoded package
>> "resource_management.libraries.functions".
>>
>> In 2.2 i was able to use param exclude_packages in install method. How i
>> can achieve same functionality in 2.4? Did i miss something?
>>
>> Thanks for help.
>>
>> [1]
>> I have list of all packages in metainfo.xml
>> For example:
>> <osSpecifics>
>> <osSpecific>
>> <osFamily>ubuntu14</osFamily>
>> <packages>
>> <package>
>> <name>supervisor</name>
>> <condition>should_install_supervisor</condition>
>> </package>
>> <package>
>> <name>apache</name>
>> <condition>should_install_apache</condition>
>> </package>
>> </packages>
>> </<osSpecific>
>> </osSpecifics>
>>
>> def should_install_supervisor():
>> config = Script.get_config()
>> # _has_applicable_local_component is copied
>> from ambari-common/src/main/python/resource_management/libra
>> ries/functions/package_conditions.py
>> return _has_applicable_local_component(config, ["MY_SERVICE"])
>>
>> [3] https://github.com/apache/ambari/blob/release-2.4.2/ambari-
>> common/src/main/python/resource_management/libraries/script/
>> script.py#L592
>>
>> --
>> Save The World - http://www.worldcommunitygrid.org/
>> http://www.worldcommunitygrid.org/stat/viewMemberInfo.do?userName=LesTR
>>
>> LesTR
>>
>
>
>
> --
> Save The World - http://www.worldcommunitygrid.org/
> http://www.worldcommunitygrid.org/stat/viewMemberInfo.do?userName=LesTR
>
> LesTR
>
--
Save The World - http://www.worldcommunitygrid.org/
http://www.worldcommunitygrid.org/stat/viewMemberInfo.do?userName=LesTR
LesTR
Re: Ambari 2.4 - custom service and condition package instalation
Posted by Lukáš Drbal <lu...@gmail.com>.
Hi Alejandro,
thank you for response! I definitely don't, because it means change ambari
source code just because adding new service and this wasn't needed in 2.2.
Am i correct?
If yes, that's strange :-(
Lukas
On Mon, Jan 9, 2017 at 8:16 PM, Alejandro Fernandez <
afernandez@hortonworks.com> wrote:
> Hi Lukas,
>
> It's not finding the method names "should_install_supervisor", "
> should_install_apache"
>
> Did you add them to this list?
>
> __all__ = ["is_lzo_enabled", "should_install_phoenix", "should_install_ams_collector", "should_install_ams_grafana",
> "should_install_mysql", "should_install_mysql_connector", "should_install_ranger_tagsync"]
>
>
> Thanks,
>
> Alejandro
>
> From: Lukáš Drbal <lu...@gmail.com>
> Reply-To: "user@ambari.apache.org" <us...@ambari.apache.org>
> Date: Monday, January 9, 2017 at 8:17 AM
> To: "user@ambari.apache.org" <us...@ambari.apache.org>
> Subject: Ambari 2.4 - custom service and condition package instalation
>
> Hi,
>
> i have custom service which provides more components [1].
> All components are installed from deb packages and i don't wanna install
> all packages on all nodes (just component specific) so i try to use
> condition [2] but it ends with error:
> "resource_management.core.exceptions.Fail: Condition with name
> 'should_install_confluent_supervisor', when installing package
> supervisor. Please check package_conditions.py."
>
> I check source code and it look's [3] like there is hardcoded package
> "resource_management.libraries.functions".
>
> In 2.2 i was able to use param exclude_packages in install method. How i
> can achieve same functionality in 2.4? Did i miss something?
>
> Thanks for help.
>
> [1]
> I have list of all packages in metainfo.xml
> For example:
> <osSpecifics>
> <osSpecific>
> <osFamily>ubuntu14</osFamily>
> <packages>
> <package>
> <name>supervisor</name>
> <condition>should_install_supervisor</condition>
> </package>
> <package>
> <name>apache</name>
> <condition>should_install_apache</condition>
> </package>
> </packages>
> </<osSpecific>
> </osSpecifics>
>
> def should_install_supervisor():
> config = Script.get_config()
> # _has_applicable_local_component is copied
> from ambari-common/src/main/python/resource_management/
> libraries/functions/package_conditions.py
> return _has_applicable_local_component(config, ["MY_SERVICE"])
>
> [3] https://github.com/apache/ambari/blob/release-2.4.2/
> ambari-common/src/main/python/resource_management/libraries/
> script/script.py#L592
>
> --
> Save The World - http://www.worldcommunitygrid.org/
> http://www.worldcommunitygrid.org/stat/viewMemberInfo.do?userName=LesTR
>
> LesTR
>
--
Save The World - http://www.worldcommunitygrid.org/
http://www.worldcommunitygrid.org/stat/viewMemberInfo.do?userName=LesTR
LesTR
Re: Ambari 2.4 - custom service and condition package instalation
Posted by Alejandro Fernandez <af...@hortonworks.com>.
Hi Lukas,
It's not finding the method names "should_install_supervisor", "should_install_apache"
Did you add them to this list?
__all__ = ["is_lzo_enabled", "should_install_phoenix", "should_install_ams_collector", "should_install_ams_grafana",
"should_install_mysql", "should_install_mysql_connector", "should_install_ranger_tagsync"]
Thanks,
Alejandro
From: Lukáš Drbal <lu...@gmail.com>>
Reply-To: "user@ambari.apache.org<ma...@ambari.apache.org>" <us...@ambari.apache.org>>
Date: Monday, January 9, 2017 at 8:17 AM
To: "user@ambari.apache.org<ma...@ambari.apache.org>" <us...@ambari.apache.org>>
Subject: Ambari 2.4 - custom service and condition package instalation
Hi,
i have custom service which provides more components [1].
All components are installed from deb packages and i don't wanna install all packages on all nodes (just component specific) so i try to use condition [2] but it ends with error:
"resource_management.core.exceptions.Fail: Condition with name 'should_install_confluent_supervisor', when installing package supervisor. Please check package_conditions.py."
I check source code and it look's [3] like there is hardcoded package "resource_management.libraries.functions".
In 2.2 i was able to use param exclude_packages in install method. How i can achieve same functionality in 2.4? Did i miss something?
Thanks for help.
[1]
I have list of all packages in metainfo.xml
For example:
<osSpecifics>
<osSpecific>
<osFamily>ubuntu14</osFamily>
<packages>
<package>
<name>supervisor</name>
<condition>should_install_supervisor</condition>
</package>
<package>
<name>apache</name>
<condition>should_install_apache</condition>
</package>
</packages>
</<osSpecific>
</osSpecifics>
def should_install_supervisor():
config = Script.get_config()
# _has_applicable_local_component is copied from ambari-common/src/main/python/resource_management/libraries/functions/package_conditions.py
return _has_applicable_local_component(config, ["MY_SERVICE"])
[3] https://github.com/apache/ambari/blob/release-2.4.2/ambari-common/src/main/python/resource_management/libraries/script/script.py#L592
--
Save The World - http://www.worldcommunitygrid.org/
http://www.worldcommunitygrid.org/stat/viewMemberInfo.do?userName=LesTR
LesTR