You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@ambari.apache.org by Di Li <di...@ca.ibm.com> on 2017/03/14 01:20:53 UTC
Review Request 57579: AMBARI-20400 Yarn should not copy Tez and
Slider tar ball if Tez and Sliders are not installed on the cluster
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57579/
-----------------------------------------------------------
Review request for Ambari and Alejandro Fernandez.
Bugs: AMBARI-20400
https://issues.apache.org/jira/browse/AMBARI-20400
Repository: ambari
Description
-------
It's possible to install a cluster without Tez and Slider via blueprint. In this case, Yarn historyserver start logic prints two warnings about unable to copy tez and slider tar ball to HDFS.
Yarn history server should skip the copy hdfs tar ball logic for tez and slider if they are not installed.
Diffs
-----
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py 8f5d380
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py 03d3fb2
ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/package/scripts/historyserver.py 8f5d380
ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/package/scripts/params_linux.py d88f518
ambari-server/src/test/python/stacks/2.0.6/configs/default.json fa7419f
Diff: https://reviews.apache.org/r/57579/diff/1/
Testing
-------
unit tests
Thanks,
Di Li
Re: Review Request 57579: AMBARI-20400 Yarn should not copy Tez and
Slider tar ball if Tez and Sliders are not installed on the cluster
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57579/#review169306
-----------------------------------------------------------
Ship it!
Ship It!
- Alejandro Fernandez
On March 17, 2017, 12:53 p.m., Di Li wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57579/
> -----------------------------------------------------------
>
> (Updated March 17, 2017, 12:53 p.m.)
>
>
> Review request for Ambari and Alejandro Fernandez.
>
>
> Bugs: AMBARI-20400
> https://issues.apache.org/jira/browse/AMBARI-20400
>
>
> Repository: ambari
>
>
> Description
> -------
>
> It's possible to install a cluster without Tez and Slider via blueprint. In this case, Yarn historyserver start logic prints two warnings about unable to copy tez and slider tar ball to HDFS.
> Yarn history server should skip the copy hdfs tar ball logic for tez and slider if they are not installed.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py 31a9be4
> ambari-server/src/test/python/stacks/2.0.6/configs/default.json fa7419f
>
>
> Diff: https://reviews.apache.org/r/57579/diff/2/
>
>
> Testing
> -------
>
> unit tests
>
>
> Thanks,
>
> Di Li
>
>
Re: Review Request 57579: AMBARI-20400 Yarn should not copy Tez and
Slider tar ball if Tez and Sliders are not installed on the cluster
Posted by Di Li <di...@ca.ibm.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57579/
-----------------------------------------------------------
(Updated March 30, 2017, 1:49 a.m.)
Review request for Ambari and Alejandro Fernandez.
Bugs: AMBARI-20400
https://issues.apache.org/jira/browse/AMBARI-20400
Repository: ambari
Description
-------
It's possible to install a cluster without Tez and Slider via blueprint. In this case, Yarn historyserver start logic prints two warnings about unable to copy tez and slider tar ball to HDFS.
Yarn history server should skip the copy hdfs tar ball logic for tez and slider if they are not installed.
Diffs (updated)
-----
ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py 31a9be4
ambari-server/src/test/python/stacks/2.0.6/configs/default.json fa7419f
Diff: https://reviews.apache.org/r/57579/diff/3/
Changes: https://reviews.apache.org/r/57579/diff/2-3/
Testing
-------
unit tests
Thanks,
Di Li
Re: Review Request 57579: AMBARI-20400 Yarn should not copy Tez and
Slider tar ball if Tez and Sliders are not installed on the cluster
Posted by Di Li <di...@ca.ibm.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57579/
-----------------------------------------------------------
(Updated March 17, 2017, 12:53 p.m.)
Review request for Ambari and Alejandro Fernandez.
Bugs: AMBARI-20400
https://issues.apache.org/jira/browse/AMBARI-20400
Repository: ambari
Description
-------
It's possible to install a cluster without Tez and Slider via blueprint. In this case, Yarn historyserver start logic prints two warnings about unable to copy tez and slider tar ball to HDFS.
Yarn history server should skip the copy hdfs tar ball logic for tez and slider if they are not installed.
Diffs (updated)
-----
ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py 31a9be4
ambari-server/src/test/python/stacks/2.0.6/configs/default.json fa7419f
Diff: https://reviews.apache.org/r/57579/diff/2/
Changes: https://reviews.apache.org/r/57579/diff/1-2/
Testing
-------
unit tests
Thanks,
Di Li
Re: Review Request 57579: AMBARI-20400 Yarn should not copy Tez and
Slider tar ball if Tez and Sliders are not installed on the cluster
Posted by Di Li <di...@ca.ibm.com>.
> On March 15, 2017, 5:48 p.m., Alejandro Fernandez wrote:
> > Can we do the same thing in all places where tarballs are copied.
> > I know Hive attempts to copy sqoop tarball, but it can potentially fail if Sqoop is not installed as a service.
> >
> > Perhaps we should actually check the topology for # of say Tez/Sqoop/Slider clients as opposed to just the existence of the config.
hello Alejandro,
I refactored the code to do the check in copy_tarball.py instead of in each service. I think this centralized implementation makes it easier to maintain. The check is against if clients are installed on the host.
Please review the new patch.
Thanks.
- Di
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57579/#review169034
-----------------------------------------------------------
On March 17, 2017, 12:53 p.m., Di Li wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57579/
> -----------------------------------------------------------
>
> (Updated March 17, 2017, 12:53 p.m.)
>
>
> Review request for Ambari and Alejandro Fernandez.
>
>
> Bugs: AMBARI-20400
> https://issues.apache.org/jira/browse/AMBARI-20400
>
>
> Repository: ambari
>
>
> Description
> -------
>
> It's possible to install a cluster without Tez and Slider via blueprint. In this case, Yarn historyserver start logic prints two warnings about unable to copy tez and slider tar ball to HDFS.
> Yarn history server should skip the copy hdfs tar ball logic for tez and slider if they are not installed.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py 31a9be4
> ambari-server/src/test/python/stacks/2.0.6/configs/default.json fa7419f
>
>
> Diff: https://reviews.apache.org/r/57579/diff/2/
>
>
> Testing
> -------
>
> unit tests
>
>
> Thanks,
>
> Di Li
>
>
Re: Review Request 57579: AMBARI-20400 Yarn should not copy Tez and
Slider tar ball if Tez and Sliders are not installed on the cluster
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57579/#review169034
-----------------------------------------------------------
Can we do the same thing in all places where tarballs are copied.
I know Hive attempts to copy sqoop tarball, but it can potentially fail if Sqoop is not installed as a service.
Perhaps we should actually check the topology for # of say Tez/Sqoop/Slider clients as opposed to just the existence of the config.
- Alejandro Fernandez
On March 14, 2017, 1:20 a.m., Di Li wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57579/
> -----------------------------------------------------------
>
> (Updated March 14, 2017, 1:20 a.m.)
>
>
> Review request for Ambari and Alejandro Fernandez.
>
>
> Bugs: AMBARI-20400
> https://issues.apache.org/jira/browse/AMBARI-20400
>
>
> Repository: ambari
>
>
> Description
> -------
>
> It's possible to install a cluster without Tez and Slider via blueprint. In this case, Yarn historyserver start logic prints two warnings about unable to copy tez and slider tar ball to HDFS.
> Yarn history server should skip the copy hdfs tar ball logic for tez and slider if they are not installed.
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py 8f5d380
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py 03d3fb2
> ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/package/scripts/historyserver.py 8f5d380
> ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/package/scripts/params_linux.py d88f518
> ambari-server/src/test/python/stacks/2.0.6/configs/default.json fa7419f
>
>
> Diff: https://reviews.apache.org/r/57579/diff/1/
>
>
> Testing
> -------
>
> unit tests
>
>
> Thanks,
>
> Di Li
>
>