You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@ambari.apache.org by Nate Cole <nc...@hortonworks.com> on 2017/11/03 02:59:38 UTC
Review Request 63535: Send ClusterSummary to install_packages and
conf-select exclusion
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63535/
-----------------------------------------------------------
Review request for Ambari, Dmitro Lisnichenko and Jonathan Hurley.
Bugs: AMBARI-22360
https://issues.apache.org/jira/browse/AMBARI-22360
Repository: ambari
Description
-------
Pass the ClusterVersionSummary object with install_packages.
- {{conf-select}} should be called as normal for {{STANDARD}} upgrades and component installations / host installs
- {{conf-select}} should only be invoked for the services involved in a {{PATCH}}/{{MAINT}} upgrade
-- The exception here is that hadoop shared components (HDFS, YARN, MapR) can never be {{conf-select}}'d in this case.
Diffs
-----
ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py c89e7675a4
ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java b3f91304c8
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProvider.java a20658304f
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostStackVersionResourceProvider.java b68dc22093
ambari-server/src/main/java/org/apache/ambari/server/state/repository/ClusterVersionSummary.java e9d99208e5
ambari-server/src/main/java/org/apache/ambari/server/state/repository/ServiceVersionSummary.java 29505c832e
ambari-server/src/main/java/org/apache/ambari/server/state/repository/VersionDefinitionXml.java 7944de8e8c
ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java 19019502dd
ambari-server/src/main/resources/custom_actions/scripts/install_packages.py 8ba22d263b
ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_packages.json 20b12a97e5
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProviderTest.java 51979f6dee
ambari-server/src/test/python/stacks/2.2/common/test_conf_select.py 92dd634cca
Diff: https://reviews.apache.org/r/63535/diff/1/
Testing
-------
Manual. Unit tests pending.
Thanks,
Nate Cole
Re: Review Request 63535: Send ClusterSummary to install_packages and
conf-select exclusion
Posted by Nate Cole <nc...@hortonworks.com>.
> On Nov. 3, 2017, 10:19 a.m., Jonathan Hurley wrote:
> > ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
> > Lines 310-312 (patched)
> > <https://reviews.apache.org/r/63535/diff/1/?file=1879724#file1879724line310>
> >
> > Should we allow this method to just be "the method" to use to get packages whether or not you're in an upgrade? That way, the dependendant code can just ask it - it would return all packages in the event that there is no PATCH/MAINT ...
Wasn't really interested in changing code that is working already when this is only for the install_packages flow.
> On Nov. 3, 2017, 10:19 a.m., Jonathan Hurley wrote:
> > ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
> > Lines 321-323 (patched)
> > <https://reviews.apache.org/r/63535/diff/1/?file=1879724#file1879724line321>
> >
> > Just trying to think if this is truly a warning. If iterating over a cluster where you have non-upgradable services, would this warning get printed out? It's fine to say that for Service X, there are no packages ... but should it be a warning?
ok
> On Nov. 3, 2017, 10:19 a.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
> > Lines 416 (patched)
> > <https://reviews.apache.org/r/63535/diff/1/?file=1879731#file1879731line416>
> >
> > Should this string be a part of KeyNames, so that its centralized?
ok
> On Nov. 3, 2017, 10:19 a.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_packages.json
> > Lines 1190 (patched)
> > <https://reviews.apache.org/r/63535/diff/1/?file=1879733#file1879733line1190>
> >
> > Do you need to adjust the stack_packages for BigInsights as well?
> >
> > Also - when you cherry pick, don't forget that there is a stack packages under HDP 3.0 in trunk...
To my knowledge, IOP clusters aren't candidates for patching and would never get this structure. So no.
Ok on 3.0
- Nate
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63535/#review190028
-----------------------------------------------------------
On Nov. 2, 2017, 10:59 p.m., Nate Cole wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63535/
> -----------------------------------------------------------
>
> (Updated Nov. 2, 2017, 10:59 p.m.)
>
>
> Review request for Ambari, Dmitro Lisnichenko and Jonathan Hurley.
>
>
> Bugs: AMBARI-22360
> https://issues.apache.org/jira/browse/AMBARI-22360
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Pass the ClusterVersionSummary object with install_packages.
>
> - {{conf-select}} should be called as normal for {{STANDARD}} upgrades and component installations / host installs
> - {{conf-select}} should only be invoked for the services involved in a {{PATCH}}/{{MAINT}} upgrade
> -- The exception here is that hadoop shared components (HDFS, YARN, MapR) can never be {{conf-select}}'d in this case.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py c89e7675a4
> ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java b3f91304c8
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProvider.java a20658304f
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostStackVersionResourceProvider.java b68dc22093
> ambari-server/src/main/java/org/apache/ambari/server/state/repository/ClusterVersionSummary.java e9d99208e5
> ambari-server/src/main/java/org/apache/ambari/server/state/repository/ServiceVersionSummary.java 29505c832e
> ambari-server/src/main/java/org/apache/ambari/server/state/repository/VersionDefinitionXml.java 7944de8e8c
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java 19019502dd
> ambari-server/src/main/resources/custom_actions/scripts/install_packages.py 8ba22d263b
> ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_packages.json 20b12a97e5
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProviderTest.java 51979f6dee
> ambari-server/src/test/python/stacks/2.2/common/test_conf_select.py 92dd634cca
>
>
> Diff: https://reviews.apache.org/r/63535/diff/1/
>
>
> Testing
> -------
>
> Manual. Unit tests:
>
> [INFO] Results:
> [INFO]
> [WARNING] Tests run: 4922, Failures: 0, Errors: 0, Skipped: 34
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 24:46.046s
> [INFO] Finished at: Thu Nov 02 23:22:42 EDT 2017
> [INFO] Final Memory: 85M/1940M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Nate Cole
>
>
Re: Review Request 63535: Send ClusterSummary to install_packages and
conf-select exclusion
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63535/#review190028
-----------------------------------------------------------
Fix it, then Ship it!
ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
Lines 310-312 (patched)
<https://reviews.apache.org/r/63535/#comment267280>
Should we allow this method to just be "the method" to use to get packages whether or not you're in an upgrade? That way, the dependendant code can just ask it - it would return all packages in the event that there is no PATCH/MAINT ...
ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
Lines 321-323 (patched)
<https://reviews.apache.org/r/63535/#comment267281>
Just trying to think if this is truly a warning. If iterating over a cluster where you have non-upgradable services, would this warning get printed out? It's fine to say that for Service X, there are no packages ... but should it be a warning?
ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
Lines 416 (patched)
<https://reviews.apache.org/r/63535/#comment267279>
Should this string be a part of KeyNames, so that its centralized?
ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_packages.json
Lines 1190 (patched)
<https://reviews.apache.org/r/63535/#comment267278>
Do you need to adjust the stack_packages for BigInsights as well?
Also - when you cherry pick, don't forget that there is a stack packages under HDP 3.0 in trunk...
- Jonathan Hurley
On Nov. 2, 2017, 10:59 p.m., Nate Cole wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63535/
> -----------------------------------------------------------
>
> (Updated Nov. 2, 2017, 10:59 p.m.)
>
>
> Review request for Ambari, Dmitro Lisnichenko and Jonathan Hurley.
>
>
> Bugs: AMBARI-22360
> https://issues.apache.org/jira/browse/AMBARI-22360
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Pass the ClusterVersionSummary object with install_packages.
>
> - {{conf-select}} should be called as normal for {{STANDARD}} upgrades and component installations / host installs
> - {{conf-select}} should only be invoked for the services involved in a {{PATCH}}/{{MAINT}} upgrade
> -- The exception here is that hadoop shared components (HDFS, YARN, MapR) can never be {{conf-select}}'d in this case.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py c89e7675a4
> ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java b3f91304c8
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProvider.java a20658304f
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostStackVersionResourceProvider.java b68dc22093
> ambari-server/src/main/java/org/apache/ambari/server/state/repository/ClusterVersionSummary.java e9d99208e5
> ambari-server/src/main/java/org/apache/ambari/server/state/repository/ServiceVersionSummary.java 29505c832e
> ambari-server/src/main/java/org/apache/ambari/server/state/repository/VersionDefinitionXml.java 7944de8e8c
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java 19019502dd
> ambari-server/src/main/resources/custom_actions/scripts/install_packages.py 8ba22d263b
> ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_packages.json 20b12a97e5
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProviderTest.java 51979f6dee
> ambari-server/src/test/python/stacks/2.2/common/test_conf_select.py 92dd634cca
>
>
> Diff: https://reviews.apache.org/r/63535/diff/1/
>
>
> Testing
> -------
>
> Manual. Unit tests:
>
> [INFO] Results:
> [INFO]
> [WARNING] Tests run: 4922, Failures: 0, Errors: 0, Skipped: 34
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 24:46.046s
> [INFO] Finished at: Thu Nov 02 23:22:42 EDT 2017
> [INFO] Final Memory: 85M/1940M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Nate Cole
>
>
Re: Review Request 63535: Send ClusterSummary to install_packages and
conf-select exclusion
Posted by Dmitro Lisnichenko <dl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63535/#review190052
-----------------------------------------------------------
Ship it!
Ship It!
- Dmitro Lisnichenko
On Nov. 3, 2017, 4:59 a.m., Nate Cole wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63535/
> -----------------------------------------------------------
>
> (Updated Nov. 3, 2017, 4:59 a.m.)
>
>
> Review request for Ambari, Dmitro Lisnichenko and Jonathan Hurley.
>
>
> Bugs: AMBARI-22360
> https://issues.apache.org/jira/browse/AMBARI-22360
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Pass the ClusterVersionSummary object with install_packages.
>
> - {{conf-select}} should be called as normal for {{STANDARD}} upgrades and component installations / host installs
> - {{conf-select}} should only be invoked for the services involved in a {{PATCH}}/{{MAINT}} upgrade
> -- The exception here is that hadoop shared components (HDFS, YARN, MapR) can never be {{conf-select}}'d in this case.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py c89e7675a4
> ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java b3f91304c8
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProvider.java a20658304f
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostStackVersionResourceProvider.java b68dc22093
> ambari-server/src/main/java/org/apache/ambari/server/state/repository/ClusterVersionSummary.java e9d99208e5
> ambari-server/src/main/java/org/apache/ambari/server/state/repository/ServiceVersionSummary.java 29505c832e
> ambari-server/src/main/java/org/apache/ambari/server/state/repository/VersionDefinitionXml.java 7944de8e8c
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java 19019502dd
> ambari-server/src/main/resources/custom_actions/scripts/install_packages.py 8ba22d263b
> ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_packages.json 20b12a97e5
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProviderTest.java 51979f6dee
> ambari-server/src/test/python/stacks/2.2/common/test_conf_select.py 92dd634cca
>
>
> Diff: https://reviews.apache.org/r/63535/diff/1/
>
>
> Testing
> -------
>
> Manual. Unit tests:
>
> [INFO] Results:
> [INFO]
> [WARNING] Tests run: 4922, Failures: 0, Errors: 0, Skipped: 34
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 24:46.046s
> [INFO] Finished at: Thu Nov 02 23:22:42 EDT 2017
> [INFO] Final Memory: 85M/1940M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Nate Cole
>
>