You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Alejandro Fernandez <af...@hortonworks.com> on 2015/05/09 04:12:40 UTC
Review Request 34010: RU wrong version of tez.tar.gz uploaded to HDFS
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34010/
-----------------------------------------------------------
Review request for Ambari, Andrew Onischuk, Jonathan Hurley, Sumit Mohanty, and Sid Wagle.
Bugs: AMBARI-11039
https://issues.apache.org/jira/browse/AMBARI-11039
Repository: ambari
Description
-------
Summary: Customers with Ambari 2.0.0 that performed an RU from say HDP 2.2.0.0 to 2.2.4.2 will not have the correct version of tez.tar.gz uploaded to HDFS.
Details: During HiveServer2 Restart in RU, it copied to wrong tez.tar.gz to HDFS (because the symlink in /usr/hdp/current/tez is still pointing to the old version at this time).
The reason why Service Checks pass is that these tarballs are compatible enough with each other. However, during RU from 2.2.* to 2.3.*, this is likely to break.
Impact: Applications that depend on this tarball (Pig, Tez, Hive) will be running with older classes. This does not appear to be breaking in HDP 2.2, otherwise customers would have reported it. However, it certainly does have to be fixed in Ambari 2.1.0 because tez.tar.gz has more significant changes and does encounter a version incompatibility error.
Diffs
-----
ambari-common/src/main/python/resource_management/libraries/functions/dynamic_variable_interpretation.py 1e70219
Diff: https://reviews.apache.org/r/34010/diff/
Testing
-------
Created a 3-node CentOS 6 cluster
Installed Ambari 2.0.1
Copied scripts
Installed HDP 2.2.0.0 GA with HDFS, YARN, MR, Hive, ZK, Tez, Pig
Added NameNode HA
Performed an RU to HDP 2.2.4.2-2, which copied the correct tarball to HDFS.
Python unit tests passed,
----------------------------------------------------------------------
Total run:641
Total errors:0
Total failures:0
OK
Thanks,
Alejandro Fernandez
Re: Review Request 34010: RU wrong version of tez.tar.gz uploaded to
HDFS
Posted by Alejandro Fernandez <af...@hortonworks.com>.
> On May 9, 2015, 8:23 a.m., Andrew Onischuk wrote:
> > ambari-common/src/main/python/resource_management/libraries/functions/dynamic_variable_interpretation.py, line 61
> > <https://reviews.apache.org/r/34010/diff/1/?file=954323#file954323line61>
> >
> > Hardcoded schema seems to be breaking support of wasb:// and glusterfs:// filesystems
Good catch, just reuploaded patch.
- Alejandro
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34010/#review83140
-----------------------------------------------------------
On May 9, 2015, 7:48 p.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34010/
> -----------------------------------------------------------
>
> (Updated May 9, 2015, 7:48 p.m.)
>
>
> Review request for Ambari, Andrew Onischuk, Jonathan Hurley, Sumit Mohanty, and Sid Wagle.
>
>
> Bugs: AMBARI-11039
> https://issues.apache.org/jira/browse/AMBARI-11039
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Summary: Customers with Ambari 2.0.0 that performed an RU from say HDP 2.2.0.0 to 2.2.4.2 will not have the correct version of tez.tar.gz uploaded to HDFS.
>
> Details: During HiveServer2 Restart in RU, it copied to wrong tez.tar.gz to HDFS (because the symlink in /usr/hdp/current/tez is still pointing to the old version at this time).
> The reason why Service Checks pass is that these tarballs are compatible enough with each other. However, during RU from 2.2.* to 2.3.*, this is likely to break.
>
> Impact: Applications that depend on this tarball (Pig, Tez, Hive) will be running with older classes. This does not appear to be breaking in HDP 2.2, otherwise customers would have reported it. However, it certainly does have to be fixed in Ambari 2.1.0 because tez.tar.gz has more significant changes and does encounter a version incompatibility error.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/dynamic_variable_interpretation.py 1e70219
>
> Diff: https://reviews.apache.org/r/34010/diff/
>
>
> Testing
> -------
>
> Created a 3-node CentOS 6 cluster
> Installed Ambari 2.0.1
> Copied scripts
> Installed HDP 2.2.0.0 GA with HDFS, YARN, MR, Hive, ZK, Tez, Pig
> Added NameNode HA
> Performed an RU to HDP 2.2.4.2-2, which copied the correct tarball to HDFS.
>
> Python unit tests passed,
>
> ----------------------------------------------------------------------
> Total run:641
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 34010: RU wrong version of tez.tar.gz uploaded to
HDFS
Posted by Andrew Onischuk <ao...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34010/#review83140
-----------------------------------------------------------
ambari-common/src/main/python/resource_management/libraries/functions/dynamic_variable_interpretation.py
<https://reviews.apache.org/r/34010/#comment134019>
Hardcoded schema seems to be breaking support of wasb:// and glusterfs:// filesystems
- Andrew Onischuk
On May 9, 2015, 2:12 a.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34010/
> -----------------------------------------------------------
>
> (Updated May 9, 2015, 2:12 a.m.)
>
>
> Review request for Ambari, Andrew Onischuk, Jonathan Hurley, Sumit Mohanty, and Sid Wagle.
>
>
> Bugs: AMBARI-11039
> https://issues.apache.org/jira/browse/AMBARI-11039
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Summary: Customers with Ambari 2.0.0 that performed an RU from say HDP 2.2.0.0 to 2.2.4.2 will not have the correct version of tez.tar.gz uploaded to HDFS.
>
> Details: During HiveServer2 Restart in RU, it copied to wrong tez.tar.gz to HDFS (because the symlink in /usr/hdp/current/tez is still pointing to the old version at this time).
> The reason why Service Checks pass is that these tarballs are compatible enough with each other. However, during RU from 2.2.* to 2.3.*, this is likely to break.
>
> Impact: Applications that depend on this tarball (Pig, Tez, Hive) will be running with older classes. This does not appear to be breaking in HDP 2.2, otherwise customers would have reported it. However, it certainly does have to be fixed in Ambari 2.1.0 because tez.tar.gz has more significant changes and does encounter a version incompatibility error.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/dynamic_variable_interpretation.py 1e70219
>
> Diff: https://reviews.apache.org/r/34010/diff/
>
>
> Testing
> -------
>
> Created a 3-node CentOS 6 cluster
> Installed Ambari 2.0.1
> Copied scripts
> Installed HDP 2.2.0.0 GA with HDFS, YARN, MR, Hive, ZK, Tez, Pig
> Added NameNode HA
> Performed an RU to HDP 2.2.4.2-2, which copied the correct tarball to HDFS.
>
> Python unit tests passed,
>
> ----------------------------------------------------------------------
> Total run:641
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 34010: RU wrong version of tez.tar.gz uploaded to
HDFS
Posted by Andrew Onischuk <ao...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34010/#review83138
-----------------------------------------------------------
Ship it!
Ship It!
- Andrew Onischuk
On May 9, 2015, 2:12 a.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34010/
> -----------------------------------------------------------
>
> (Updated May 9, 2015, 2:12 a.m.)
>
>
> Review request for Ambari, Andrew Onischuk, Jonathan Hurley, Sumit Mohanty, and Sid Wagle.
>
>
> Bugs: AMBARI-11039
> https://issues.apache.org/jira/browse/AMBARI-11039
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Summary: Customers with Ambari 2.0.0 that performed an RU from say HDP 2.2.0.0 to 2.2.4.2 will not have the correct version of tez.tar.gz uploaded to HDFS.
>
> Details: During HiveServer2 Restart in RU, it copied to wrong tez.tar.gz to HDFS (because the symlink in /usr/hdp/current/tez is still pointing to the old version at this time).
> The reason why Service Checks pass is that these tarballs are compatible enough with each other. However, during RU from 2.2.* to 2.3.*, this is likely to break.
>
> Impact: Applications that depend on this tarball (Pig, Tez, Hive) will be running with older classes. This does not appear to be breaking in HDP 2.2, otherwise customers would have reported it. However, it certainly does have to be fixed in Ambari 2.1.0 because tez.tar.gz has more significant changes and does encounter a version incompatibility error.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/dynamic_variable_interpretation.py 1e70219
>
> Diff: https://reviews.apache.org/r/34010/diff/
>
>
> Testing
> -------
>
> Created a 3-node CentOS 6 cluster
> Installed Ambari 2.0.1
> Copied scripts
> Installed HDP 2.2.0.0 GA with HDFS, YARN, MR, Hive, ZK, Tez, Pig
> Added NameNode HA
> Performed an RU to HDP 2.2.4.2-2, which copied the correct tarball to HDFS.
>
> Python unit tests passed,
>
> ----------------------------------------------------------------------
> Total run:641
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 34010: RU wrong version of tez.tar.gz uploaded to
HDFS
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34010/
-----------------------------------------------------------
(Updated May 9, 2015, 7:48 p.m.)
Review request for Ambari, Andrew Onischuk, Jonathan Hurley, Sumit Mohanty, and Sid Wagle.
Changes
-------
Addressed comments.
Bugs: AMBARI-11039
https://issues.apache.org/jira/browse/AMBARI-11039
Repository: ambari
Description
-------
Summary: Customers with Ambari 2.0.0 that performed an RU from say HDP 2.2.0.0 to 2.2.4.2 will not have the correct version of tez.tar.gz uploaded to HDFS.
Details: During HiveServer2 Restart in RU, it copied to wrong tez.tar.gz to HDFS (because the symlink in /usr/hdp/current/tez is still pointing to the old version at this time).
The reason why Service Checks pass is that these tarballs are compatible enough with each other. However, during RU from 2.2.* to 2.3.*, this is likely to break.
Impact: Applications that depend on this tarball (Pig, Tez, Hive) will be running with older classes. This does not appear to be breaking in HDP 2.2, otherwise customers would have reported it. However, it certainly does have to be fixed in Ambari 2.1.0 because tez.tar.gz has more significant changes and does encounter a version incompatibility error.
Diffs (updated)
-----
ambari-common/src/main/python/resource_management/libraries/functions/dynamic_variable_interpretation.py 1e70219
Diff: https://reviews.apache.org/r/34010/diff/
Testing
-------
Created a 3-node CentOS 6 cluster
Installed Ambari 2.0.1
Copied scripts
Installed HDP 2.2.0.0 GA with HDFS, YARN, MR, Hive, ZK, Tez, Pig
Added NameNode HA
Performed an RU to HDP 2.2.4.2-2, which copied the correct tarball to HDFS.
Python unit tests passed,
----------------------------------------------------------------------
Total run:641
Total errors:0
Total failures:0
OK
Thanks,
Alejandro Fernandez