You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@ambari.apache.org by Renjith Kamath <re...@gmail.com> on 2017/01/19 12:41:56 UTC

Review Request 55717: Cannot start Zeppelin notebook if Hive is not installed

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55717/
-----------------------------------------------------------

Review request for Ambari, Alejandro Fernandez, DIPAYAN BHOWMICK, Jayush Luniya, Prabhjyot Singh, Rohit Choudhary, and Sumit Mohanty.


Bugs: AMBARI-19627
    https://issues.apache.org/jira/browse/AMBARI-19627


Repository: ambari


Description
-------

Hive is a dependency only for Spark ThriftServer, not for other Spark components or Zeppelin. Yet, Zeppelin fails to start if Hive is not installed:

Error:
resource_management.core.exceptions.Fail: StaticFile('/etc/spark/conf/hive-site.xml') Source file /etc/spark/conf/hive-site.xml is not found


This review is for branch-2.5 patch. There is a separate patch attached in jira for trunk.


Diffs
-----

  ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/master.py ac56aa2 

Diff: https://reviews.apache.org/r/55717/diff/


Testing
-------

tested on CentOS and passed unit tests on trunk


Thanks,

Renjith Kamath

Re: Review Request 55717: Cannot start Zeppelin notebook if Hive is not installed

Posted by Prabhjyot Singh <pr...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55717/#review162275
-----------------------------------------------------------


Ship it!




Ship It!

- Prabhjyot Singh


On Jan. 19, 2017, 12:41 p.m., Renjith Kamath wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55717/
> -----------------------------------------------------------
> 
> (Updated Jan. 19, 2017, 12:41 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, DIPAYAN BHOWMICK, Jayush Luniya, Prabhjyot Singh, Rohit Choudhary, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-19627
>     https://issues.apache.org/jira/browse/AMBARI-19627
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> Hive is a dependency only for Spark ThriftServer, not for other Spark components or Zeppelin. Yet, Zeppelin fails to start if Hive is not installed:
> 
> Error:
> resource_management.core.exceptions.Fail: StaticFile('/etc/spark/conf/hive-site.xml') Source file /etc/spark/conf/hive-site.xml is not found
> 
> 
> This review is for branch-2.5 patch. There is a separate patch attached in jira for trunk.
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/master.py ac56aa2 
> 
> Diff: https://reviews.apache.org/r/55717/diff/
> 
> 
> Testing
> -------
> 
> tested on CentOS and passed unit tests on trunk
> 
> 
> Thanks,
> 
> Renjith Kamath
> 
>

Re: Review Request 55717: Cannot start Zeppelin notebook if Hive is not installed

Posted by Renjith Kamath <re...@gmail.com>.

> On Jan. 19, 2017, 1:35 p.m., Attila Doroszlai wrote:
> > ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/master.py, line 153
> > <https://reviews.apache.org/r/55717/diff/1/?file=1608974#file1608974line153>
> >
> >     Since the path is fixed (not a pattern), why is `glob.glob` needed?

removed


- Renjith


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55717/#review162277
-----------------------------------------------------------


On Jan. 19, 2017, 3:58 p.m., Renjith Kamath wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55717/
> -----------------------------------------------------------
> 
> (Updated Jan. 19, 2017, 3:58 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, DIPAYAN BHOWMICK, Jayush Luniya, Prabhjyot Singh, Rohit Choudhary, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-19627
>     https://issues.apache.org/jira/browse/AMBARI-19627
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> Hive is a dependency only for Spark ThriftServer, not for other Spark components or Zeppelin. Yet, Zeppelin fails to start if Hive is not installed:
> 
> Error:
> resource_management.core.exceptions.Fail: StaticFile('/etc/spark/conf/hive-site.xml') Source file /etc/spark/conf/hive-site.xml is not found
> 
> 
> This review is for branch-2.5 patch. There is a separate patch attached in jira for trunk.
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/master.py ac56aa2 
> 
> Diff: https://reviews.apache.org/r/55717/diff/
> 
> 
> Testing
> -------
> 
> tested on CentOS and passed unit tests on trunk
> 
> 
> Thanks,
> 
> Renjith Kamath
> 
>

Re: Review Request 55717: Cannot start Zeppelin notebook if Hive is not installed

Posted by Attila Doroszlai <ad...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55717/#review162277
-----------------------------------------------------------




ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/master.py (line 153)
<https://reviews.apache.org/r/55717/#comment233614>

    Since the path is fixed (not a pattern), why is `glob.glob` needed?


- Attila Doroszlai


On Jan. 19, 2017, 1:41 p.m., Renjith Kamath wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55717/
> -----------------------------------------------------------
> 
> (Updated Jan. 19, 2017, 1:41 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, DIPAYAN BHOWMICK, Jayush Luniya, Prabhjyot Singh, Rohit Choudhary, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-19627
>     https://issues.apache.org/jira/browse/AMBARI-19627
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> Hive is a dependency only for Spark ThriftServer, not for other Spark components or Zeppelin. Yet, Zeppelin fails to start if Hive is not installed:
> 
> Error:
> resource_management.core.exceptions.Fail: StaticFile('/etc/spark/conf/hive-site.xml') Source file /etc/spark/conf/hive-site.xml is not found
> 
> 
> This review is for branch-2.5 patch. There is a separate patch attached in jira for trunk.
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/master.py ac56aa2 
> 
> Diff: https://reviews.apache.org/r/55717/diff/
> 
> 
> Testing
> -------
> 
> tested on CentOS and passed unit tests on trunk
> 
> 
> Thanks,
> 
> Renjith Kamath
> 
>

Re: Review Request 55717: Cannot start Zeppelin notebook if Hive is not installed

Posted by Renjith Kamath <re...@gmail.com>.

> On Jan. 19, 2017, 11:21 p.m., Attila Doroszlai wrote:
> > The patches fix the problem with starting Zeppelin without Hive.
> > 
> > However, the [patch against trunk](https://issues.apache.org/jira/secure/attachment/12848330/AMBARI-19627-trunk-v2.patch) has problems with the unit tests.
> > 
> > 1. The methods where `@patch` is added need an extra parameter declared for the mock object.
> > 
> > ```
> > TypeError: assert_configure_default() takes exactly 1 argument (2 given)
> > TypeError: assert_configure_secured() takes exactly 1 argument (2 given)
> > ```
> > 
> > 2. Even after adding the missing parameters to the two methods, the unit tests fail:
> > 
> > ```
> > FAIL: test_configure_secured (test_zeppelin_master.TestZeppelinMaster)
> > ...
> > AssertionError: There were no more resources executed!
> > 
> > FAIL: test_configure_default (test_zeppelin_master.TestZeppelinMaster)
> > ...
> > AssertionError: There were no more resources executed!
> > 
> > FAIL: test_start_default (test_zeppelin_master.TestZeppelinMaster)
> > ...
> > AssertionError: 'File' != 'Execute'
> > 
> > FAIL: test_start_secured (test_zeppelin_master.TestZeppelinMaster)
> > ...
> > AssertionError: 'File' != 'Execute'
> > ```
> > 
> > The tests pass after manually creating `/etc/spark/conf/hive-site.xml`.  It doesn't seem like the mocked `os.path.exists` check is working as expected.

@adoroszlai thanks for the unit test fix https://issues.apache.org/jira/secure/attachment/12848492/AMBARI-19627-trunk-v3.patch


- Renjith


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55717/#review162358
-----------------------------------------------------------


On Jan. 19, 2017, 3:58 p.m., Renjith Kamath wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55717/
> -----------------------------------------------------------
> 
> (Updated Jan. 19, 2017, 3:58 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, DIPAYAN BHOWMICK, Jayush Luniya, Prabhjyot Singh, Rohit Choudhary, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-19627
>     https://issues.apache.org/jira/browse/AMBARI-19627
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> Hive is a dependency only for Spark ThriftServer, not for other Spark components or Zeppelin. Yet, Zeppelin fails to start if Hive is not installed:
> 
> Error:
> resource_management.core.exceptions.Fail: StaticFile('/etc/spark/conf/hive-site.xml') Source file /etc/spark/conf/hive-site.xml is not found
> 
> 
> This review is for branch-2.5 patch. There is a separate patch attached in jira for trunk.
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/master.py ac56aa2 
> 
> Diff: https://reviews.apache.org/r/55717/diff/
> 
> 
> Testing
> -------
> 
> tested on CentOS and passed unit tests on trunk
> 
> 
> Thanks,
> 
> Renjith Kamath
> 
>

Re: Review Request 55717: Cannot start Zeppelin notebook if Hive is not installed

Posted by Attila Doroszlai <ad...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55717/#review162358
-----------------------------------------------------------



The patches fix the problem with starting Zeppelin without Hive.

However, the [patch against trunk](https://issues.apache.org/jira/secure/attachment/12848330/AMBARI-19627-trunk-v2.patch) has problems with the unit tests.

1. The methods where `@patch` is added need an extra parameter declared for the mock object.

```
TypeError: assert_configure_default() takes exactly 1 argument (2 given)
TypeError: assert_configure_secured() takes exactly 1 argument (2 given)
```

2. Even after adding the missing parameters to the two methods, the unit tests fail:

```
FAIL: test_configure_secured (test_zeppelin_master.TestZeppelinMaster)
...
AssertionError: There were no more resources executed!

FAIL: test_configure_default (test_zeppelin_master.TestZeppelinMaster)
...
AssertionError: There were no more resources executed!

FAIL: test_start_default (test_zeppelin_master.TestZeppelinMaster)
...
AssertionError: 'File' != 'Execute'

FAIL: test_start_secured (test_zeppelin_master.TestZeppelinMaster)
...
AssertionError: 'File' != 'Execute'
```

The tests pass after manually creating `/etc/spark/conf/hive-site.xml`.  It doesn't seem like the mocked `os.path.exists` check is working as expected.

- Attila Doroszlai


On Jan. 19, 2017, 4:58 p.m., Renjith Kamath wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55717/
> -----------------------------------------------------------
> 
> (Updated Jan. 19, 2017, 4:58 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, DIPAYAN BHOWMICK, Jayush Luniya, Prabhjyot Singh, Rohit Choudhary, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-19627
>     https://issues.apache.org/jira/browse/AMBARI-19627
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> Hive is a dependency only for Spark ThriftServer, not for other Spark components or Zeppelin. Yet, Zeppelin fails to start if Hive is not installed:
> 
> Error:
> resource_management.core.exceptions.Fail: StaticFile('/etc/spark/conf/hive-site.xml') Source file /etc/spark/conf/hive-site.xml is not found
> 
> 
> This review is for branch-2.5 patch. There is a separate patch attached in jira for trunk.
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/master.py ac56aa2 
> 
> Diff: https://reviews.apache.org/r/55717/diff/
> 
> 
> Testing
> -------
> 
> tested on CentOS and passed unit tests on trunk
> 
> 
> Thanks,
> 
> Renjith Kamath
> 
>

Re: Review Request 55717: Cannot start Zeppelin notebook if Hive is not installed

Posted by Attila Doroszlai <ad...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55717/#review162433
-----------------------------------------------------------


Ship it!




Ship It!

- Attila Doroszlai


On Jan. 19, 2017, 4:58 p.m., Renjith Kamath wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55717/
> -----------------------------------------------------------
> 
> (Updated Jan. 19, 2017, 4:58 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, DIPAYAN BHOWMICK, Jayush Luniya, Prabhjyot Singh, Rohit Choudhary, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-19627
>     https://issues.apache.org/jira/browse/AMBARI-19627
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> Hive is a dependency only for Spark ThriftServer, not for other Spark components or Zeppelin. Yet, Zeppelin fails to start if Hive is not installed:
> 
> Error:
> resource_management.core.exceptions.Fail: StaticFile('/etc/spark/conf/hive-site.xml') Source file /etc/spark/conf/hive-site.xml is not found
> 
> 
> This review is for branch-2.5 patch. There is a separate patch attached in jira for trunk.
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/master.py ac56aa2 
> 
> Diff: https://reviews.apache.org/r/55717/diff/
> 
> 
> Testing
> -------
> 
> tested on CentOS and passed unit tests on trunk
> 
> 
> Thanks,
> 
> Renjith Kamath
> 
>

Re: Review Request 55717: Cannot start Zeppelin notebook if Hive is not installed

Posted by Renjith Kamath <re...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55717/
-----------------------------------------------------------

(Updated Jan. 19, 2017, 3:58 p.m.)


Review request for Ambari, Alejandro Fernandez, DIPAYAN BHOWMICK, Jayush Luniya, Prabhjyot Singh, Rohit Choudhary, and Sumit Mohanty.


Bugs: AMBARI-19627
    https://issues.apache.org/jira/browse/AMBARI-19627


Repository: ambari


Description
-------

Hive is a dependency only for Spark ThriftServer, not for other Spark components or Zeppelin. Yet, Zeppelin fails to start if Hive is not installed:

Error:
resource_management.core.exceptions.Fail: StaticFile('/etc/spark/conf/hive-site.xml') Source file /etc/spark/conf/hive-site.xml is not found


This review is for branch-2.5 patch. There is a separate patch attached in jira for trunk.


Diffs (updated)
-----

  ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/master.py ac56aa2 

Diff: https://reviews.apache.org/r/55717/diff/


Testing
-------

tested on CentOS and passed unit tests on trunk


Thanks,

Renjith Kamath