You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Jonathan Hurley <jh...@hortonworks.com> on 2015/05/18 19:11:42 UTC
Review Request 34363: Tez Upgrade Pack For HDP-2.2 to HDP-2.3
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34363/
-----------------------------------------------------------
Review request for Ambari, Alejandro Fernandez and Nate Cole.
Bugs: AMBARI-11220
https://issues.apache.org/jira/browse/AMBARI-11220
Repository: ambari
Description
-------
In the latest version of Tez, additional security controls ( ACLs) were introduced. To provide the same level of access control as the previous version (i.e. no access controls), add the following property to tez-site.xml:
{code}
<property>
<name>tez.am.view-acls</name>
<value>*</value>
</property>
{code}
Additionally, we are also now setting the property “tez.task.generate.counters.per.io” to true in the latest release. However, this is not a critical config that needs to be applied as part of the rolling upgrade.
Diffs
-----
ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py 7c5c005
ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/params_linux.py 1e183de
ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/service_check.py abff479
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/params.py 3031a9f
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/scripts/params.py 9d57c18
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py 889dbd1
ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.3.xml 8905561
Diff: https://reviews.apache.org/r/34363/diff/
Testing
-------
Upgrade HDP 2.2 to 2.3 with Tez + core hadoop.
Thanks,
Jonathan Hurley
Re: Review Request 34363: Tez Upgrade Pack For HDP-2.2 to HDP-2.3
Posted by Nate Cole <nc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34363/#review84179
-----------------------------------------------------------
Ship it!
Ship It!
- Nate Cole
On May 18, 2015, 1:11 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34363/
> -----------------------------------------------------------
>
> (Updated May 18, 2015, 1:11 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez and Nate Cole.
>
>
> Bugs: AMBARI-11220
> https://issues.apache.org/jira/browse/AMBARI-11220
>
>
> Repository: ambari
>
>
> Description
> -------
>
> In the latest version of Tez, additional security controls ( ACLs) were introduced. To provide the same level of access control as the previous version (i.e. no access controls), add the following property to tez-site.xml:
> {code}
> <property>
> <name>tez.am.view-acls</name>
> <value>*</value>
> </property>
> {code}
>
> Additionally, we are also now setting the property “tez.task.generate.counters.per.io” to true in the latest release. However, this is not a critical config that needs to be applied as part of the rolling upgrade.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py 7c5c005
> ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/params_linux.py 1e183de
> ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/service_check.py abff479
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/params.py 3031a9f
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/scripts/params.py 9d57c18
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py 889dbd1
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.3.xml 8905561
>
> Diff: https://reviews.apache.org/r/34363/diff/
>
>
> Testing
> -------
>
> Upgrade HDP 2.2 to 2.3 with Tez + core hadoop.
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 34363: Tez Upgrade Pack For HDP-2.2 to HDP-2.3
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34363/#review84210
-----------------------------------------------------------
Ship it!
Ship It!
- Alejandro Fernandez
On May 18, 2015, 5:11 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34363/
> -----------------------------------------------------------
>
> (Updated May 18, 2015, 5:11 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez and Nate Cole.
>
>
> Bugs: AMBARI-11220
> https://issues.apache.org/jira/browse/AMBARI-11220
>
>
> Repository: ambari
>
>
> Description
> -------
>
> In the latest version of Tez, additional security controls ( ACLs) were introduced. To provide the same level of access control as the previous version (i.e. no access controls), add the following property to tez-site.xml:
> {code}
> <property>
> <name>tez.am.view-acls</name>
> <value>*</value>
> </property>
> {code}
>
> Additionally, we are also now setting the property “tez.task.generate.counters.per.io” to true in the latest release. However, this is not a critical config that needs to be applied as part of the rolling upgrade.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py 7c5c005
> ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/params_linux.py 1e183de
> ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/service_check.py abff479
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/params.py 3031a9f
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/scripts/params.py 9d57c18
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py 889dbd1
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.3.xml 8905561
>
> Diff: https://reviews.apache.org/r/34363/diff/
>
>
> Testing
> -------
>
> Upgrade HDP 2.2 to 2.3 with Tez + core hadoop.
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 34363: Tez Upgrade Pack For HDP-2.2 to HDP-2.3
Posted by Jonathan Hurley <jh...@hortonworks.com>.
> On May 18, 2015, 4:16 p.m., Alejandro Fernandez wrote:
> > ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py, line 174
> > <https://reviews.apache.org/r/34363/diff/1/?file=962956#file962956line174>
> >
> > If current_hdp_version is None, due to some error, then this not update stack_version, so the wrong balue will be used.
> > At a minimum, we should log it.
I don't think this can be None as an exception is thrown if it can't be determined.
- Jonathan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34363/#review84206
-----------------------------------------------------------
On May 18, 2015, 1:11 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34363/
> -----------------------------------------------------------
>
> (Updated May 18, 2015, 1:11 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez and Nate Cole.
>
>
> Bugs: AMBARI-11220
> https://issues.apache.org/jira/browse/AMBARI-11220
>
>
> Repository: ambari
>
>
> Description
> -------
>
> In the latest version of Tez, additional security controls ( ACLs) were introduced. To provide the same level of access control as the previous version (i.e. no access controls), add the following property to tez-site.xml:
> {code}
> <property>
> <name>tez.am.view-acls</name>
> <value>*</value>
> </property>
> {code}
>
> Additionally, we are also now setting the property “tez.task.generate.counters.per.io” to true in the latest release. However, this is not a critical config that needs to be applied as part of the rolling upgrade.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py 7c5c005
> ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/params_linux.py 1e183de
> ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/service_check.py abff479
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/params.py 3031a9f
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/scripts/params.py 9d57c18
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py 889dbd1
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.3.xml 8905561
>
> Diff: https://reviews.apache.org/r/34363/diff/
>
>
> Testing
> -------
>
> Upgrade HDP 2.2 to 2.3 with Tez + core hadoop.
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 34363: Tez Upgrade Pack For HDP-2.2 to HDP-2.3
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34363/#review84206
-----------------------------------------------------------
ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
<https://reviews.apache.org/r/34363/#comment135351>
If current_hdp_version is None, due to some error, then this not update stack_version, so the wrong balue will be used.
At a minimum, we should log it.
- Alejandro Fernandez
On May 18, 2015, 5:11 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34363/
> -----------------------------------------------------------
>
> (Updated May 18, 2015, 5:11 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez and Nate Cole.
>
>
> Bugs: AMBARI-11220
> https://issues.apache.org/jira/browse/AMBARI-11220
>
>
> Repository: ambari
>
>
> Description
> -------
>
> In the latest version of Tez, additional security controls ( ACLs) were introduced. To provide the same level of access control as the previous version (i.e. no access controls), add the following property to tez-site.xml:
> {code}
> <property>
> <name>tez.am.view-acls</name>
> <value>*</value>
> </property>
> {code}
>
> Additionally, we are also now setting the property “tez.task.generate.counters.per.io” to true in the latest release. However, this is not a critical config that needs to be applied as part of the rolling upgrade.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py 7c5c005
> ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/params_linux.py 1e183de
> ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/service_check.py abff479
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/params.py 3031a9f
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/scripts/params.py 9d57c18
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py 889dbd1
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.3.xml 8905561
>
> Diff: https://reviews.apache.org/r/34363/diff/
>
>
> Testing
> -------
>
> Upgrade HDP 2.2 to 2.3 with Tez + core hadoop.
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 34363: Tez Upgrade Pack For HDP-2.2 to HDP-2.3
Posted by Nate Cole <nc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34363/#review84169
-----------------------------------------------------------
ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.3.xml
<https://reviews.apache.org/r/34363/#comment135297>
Dang, we should just add the ability to set multiple things. This is just going to add multiple stages for the same site file.
<task xsi:type="configure">
<type>tez-site</type>
<value key="..." value="..." />
<value key="..." value="..." />
</task>
- Nate Cole
On May 18, 2015, 1:11 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34363/
> -----------------------------------------------------------
>
> (Updated May 18, 2015, 1:11 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez and Nate Cole.
>
>
> Bugs: AMBARI-11220
> https://issues.apache.org/jira/browse/AMBARI-11220
>
>
> Repository: ambari
>
>
> Description
> -------
>
> In the latest version of Tez, additional security controls ( ACLs) were introduced. To provide the same level of access control as the previous version (i.e. no access controls), add the following property to tez-site.xml:
> {code}
> <property>
> <name>tez.am.view-acls</name>
> <value>*</value>
> </property>
> {code}
>
> Additionally, we are also now setting the property “tez.task.generate.counters.per.io” to true in the latest release. However, this is not a critical config that needs to be applied as part of the rolling upgrade.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py 7c5c005
> ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/params_linux.py 1e183de
> ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/service_check.py abff479
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/params.py 3031a9f
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/scripts/params.py 9d57c18
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py 889dbd1
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.3.xml 8905561
>
> Diff: https://reviews.apache.org/r/34363/diff/
>
>
> Testing
> -------
>
> Upgrade HDP 2.2 to 2.3 with Tez + core hadoop.
>
>
> Thanks,
>
> Jonathan Hurley
>
>