You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Matt <mi...@gmail.com> on 2015/12/22 12:23:05 UTC
Review Request 41641: Recommend PXF component to be collocated with
NAMENODE and DATANODE
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41641/
-----------------------------------------------------------
Review request for Ambari, Alexander Denissov, Alejandro Fernandez, Alexandr Antonenko, bhuvnesh chaudhary, Jaimin Jetly, jun aoki, Jayush Luniya, Lav Jain, Newton Alex, and Oleksandr Diachenko.
Bugs: AMBARI-14412
https://issues.apache.org/jira/browse/AMBARI-14412
Repository: ambari
Description
-------
It is recommended that PXF component should be installed on the same hosts which has NAMENODE or/and DATANODE.
Diffs
-----
ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 5ab05fd
ambari-server/src/test/python/stacks/2.3/common/services-pxf-hdfs.json PRE-CREATION
ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py db61f5f
Diff: https://reviews.apache.org/r/41641/diff/
Testing
-------
Added unit test in stack advisor.
Test passed.
Also tested manually.
Thanks,
Matt
Re: Review Request 41641: Recommend PXF component to be collocated
with NAMENODE and DATANODE
Posted by Newton Alex <na...@pivotal.io>.
> On Dec. 22, 2015, 8:13 p.m., Newton Alex wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py, line 43
> > <https://reviews.apache.org/r/41641/diff/1/?file=1174500#file1174500line43>
> >
> > Should we also be looking for Standby and Secondary Namenodes?
Sounds good. Please ensure that you have the ticket created for the Stanby Namnode scenario.
Thanks
- Newton
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41641/#review111642
-----------------------------------------------------------
On Dec. 22, 2015, 7:36 p.m., Matt wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41641/
> -----------------------------------------------------------
>
> (Updated Dec. 22, 2015, 7:36 p.m.)
>
>
> Review request for Ambari, Alexander Denissov, Alejandro Fernandez, Alexandr Antonenko, bhuvnesh chaudhary, Jaimin Jetly, jun aoki, Jayush Luniya, Lav Jain, Newton Alex, and Oleksandr Diachenko.
>
>
> Bugs: AMBARI-14412
> https://issues.apache.org/jira/browse/AMBARI-14412
>
>
> Repository: ambari
>
>
> Description
> -------
>
> It is recommended that PXF component should be installed on the same hosts which has NAMENODE or/and DATANODE.
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 5ab05fd
> ambari-server/src/test/python/stacks/2.3/common/services-pxf-hdfs.json PRE-CREATION
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py db61f5f
>
> Diff: https://reviews.apache.org/r/41641/diff/
>
>
> Testing
> -------
>
> Added unit test in stack advisor.
> Test passed.
>
> OK
> Running tests for stack:2.3 service:common
> Test that HAWQSTANDBY is not recommended on a single node cluster ... ok
> Test that HAWQSTANDBY is recommended on a 3-node cluster ... ok
> Test no failures when there are no HAWQ components ... ok
> **Test that PXF gets recommended on same host group which has NAMENODE or DATANODE ... ok**
> Test layout validations for HAWQ components on a 3-node cluster ... ok
> Test no failures when there are no HAWQ components on a 3-node cluster ... ok
> Test SparkTS is picked when Hive is not installed ... ok
> Test SparkTS is picked when Hive is installed ... ok
> test_noRiskyDictLookups (test_stack_advisor.TestHDP23StackAdvisor) ... ok
>
> Also tested manually.
>
>
> Thanks,
>
> Matt
>
>
Re: Review Request 41641: Recommend PXF component to be collocated
with NAMENODE and DATANODE
Posted by Matt <mi...@gmail.com>.
> On Dec. 22, 2015, 12:13 p.m., Newton Alex wrote:
> >
PXF should be co-located with an **active** Namenode and/or Datanode.
Secondary Namenode: It is not an **active** Namenode - hence there is no PXF component required on the host.
Standby Namenode: Two scenarios to consider here based on when PXF is installed on cluster (before or after enabling Namenode HA)
1. PXF is already installed on the cluster, and then Namenode HA is enabled: In this case, the HA wizard should be capable of adding a PXF component on the same host as the new Namenode (if the host does not have PXF already). This would be covered in another JIRA, which deals with the Namenode HA wizard.
2. Namenode HA is enabled, and then PXF is installed on the cluster: Both Namenodes are referred to as "NAMENODE" component. This case would be handled by stack_advisor code in line 41.
- Matt
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41641/#review111642
-----------------------------------------------------------
On Dec. 22, 2015, 11:36 a.m., Matt wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41641/
> -----------------------------------------------------------
>
> (Updated Dec. 22, 2015, 11:36 a.m.)
>
>
> Review request for Ambari, Alexander Denissov, Alejandro Fernandez, Alexandr Antonenko, bhuvnesh chaudhary, Jaimin Jetly, jun aoki, Jayush Luniya, Lav Jain, Newton Alex, and Oleksandr Diachenko.
>
>
> Bugs: AMBARI-14412
> https://issues.apache.org/jira/browse/AMBARI-14412
>
>
> Repository: ambari
>
>
> Description
> -------
>
> It is recommended that PXF component should be installed on the same hosts which has NAMENODE or/and DATANODE.
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 5ab05fd
> ambari-server/src/test/python/stacks/2.3/common/services-pxf-hdfs.json PRE-CREATION
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py db61f5f
>
> Diff: https://reviews.apache.org/r/41641/diff/
>
>
> Testing
> -------
>
> Added unit test in stack advisor.
> Test passed.
>
> OK
> Running tests for stack:2.3 service:common
> Test that HAWQSTANDBY is not recommended on a single node cluster ... ok
> Test that HAWQSTANDBY is recommended on a 3-node cluster ... ok
> Test no failures when there are no HAWQ components ... ok
> **Test that PXF gets recommended on same host group which has NAMENODE or DATANODE ... ok**
> Test layout validations for HAWQ components on a 3-node cluster ... ok
> Test no failures when there are no HAWQ components on a 3-node cluster ... ok
> Test SparkTS is picked when Hive is not installed ... ok
> Test SparkTS is picked when Hive is installed ... ok
> test_noRiskyDictLookups (test_stack_advisor.TestHDP23StackAdvisor) ... ok
>
> Also tested manually.
>
>
> Thanks,
>
> Matt
>
>
Re: Review Request 41641: Recommend PXF component to be collocated
with NAMENODE and DATANODE
Posted by Newton Alex <na...@pivotal.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41641/#review111642
-----------------------------------------------------------
ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py (line 41)
<https://reviews.apache.org/r/41641/#comment171855>
Should we also be looking for Standby and Secondary Namenodes?
- Newton Alex
On Dec. 22, 2015, 7:36 p.m., Matt wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41641/
> -----------------------------------------------------------
>
> (Updated Dec. 22, 2015, 7:36 p.m.)
>
>
> Review request for Ambari, Alexander Denissov, Alejandro Fernandez, Alexandr Antonenko, bhuvnesh chaudhary, Jaimin Jetly, jun aoki, Jayush Luniya, Lav Jain, Newton Alex, and Oleksandr Diachenko.
>
>
> Bugs: AMBARI-14412
> https://issues.apache.org/jira/browse/AMBARI-14412
>
>
> Repository: ambari
>
>
> Description
> -------
>
> It is recommended that PXF component should be installed on the same hosts which has NAMENODE or/and DATANODE.
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 5ab05fd
> ambari-server/src/test/python/stacks/2.3/common/services-pxf-hdfs.json PRE-CREATION
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py db61f5f
>
> Diff: https://reviews.apache.org/r/41641/diff/
>
>
> Testing
> -------
>
> Added unit test in stack advisor.
> Test passed.
>
> OK
> Running tests for stack:2.3 service:common
> Test that HAWQSTANDBY is not recommended on a single node cluster ... ok
> Test that HAWQSTANDBY is recommended on a 3-node cluster ... ok
> Test no failures when there are no HAWQ components ... ok
> **Test that PXF gets recommended on same host group which has NAMENODE or DATANODE ... ok**
> Test layout validations for HAWQ components on a 3-node cluster ... ok
> Test no failures when there are no HAWQ components on a 3-node cluster ... ok
> Test SparkTS is picked when Hive is not installed ... ok
> Test SparkTS is picked when Hive is installed ... ok
> test_noRiskyDictLookups (test_stack_advisor.TestHDP23StackAdvisor) ... ok
>
> Also tested manually.
>
>
> Thanks,
>
> Matt
>
>
Re: Review Request 41641: Recommend PXF component to be collocated
with NAMENODE and DATANODE
Posted by Lav Jain <lj...@pivotal.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41641/#review111643
-----------------------------------------------------------
Ship it!
Ship It!
- Lav Jain
On Dec. 22, 2015, 7:36 p.m., Matt wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41641/
> -----------------------------------------------------------
>
> (Updated Dec. 22, 2015, 7:36 p.m.)
>
>
> Review request for Ambari, Alexander Denissov, Alejandro Fernandez, Alexandr Antonenko, bhuvnesh chaudhary, Jaimin Jetly, jun aoki, Jayush Luniya, Lav Jain, Newton Alex, and Oleksandr Diachenko.
>
>
> Bugs: AMBARI-14412
> https://issues.apache.org/jira/browse/AMBARI-14412
>
>
> Repository: ambari
>
>
> Description
> -------
>
> It is recommended that PXF component should be installed on the same hosts which has NAMENODE or/and DATANODE.
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 5ab05fd
> ambari-server/src/test/python/stacks/2.3/common/services-pxf-hdfs.json PRE-CREATION
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py db61f5f
>
> Diff: https://reviews.apache.org/r/41641/diff/
>
>
> Testing
> -------
>
> Added unit test in stack advisor.
> Test passed.
>
> OK
> Running tests for stack:2.3 service:common
> Test that HAWQSTANDBY is not recommended on a single node cluster ... ok
> Test that HAWQSTANDBY is recommended on a 3-node cluster ... ok
> Test no failures when there are no HAWQ components ... ok
> **Test that PXF gets recommended on same host group which has NAMENODE or DATANODE ... ok**
> Test layout validations for HAWQ components on a 3-node cluster ... ok
> Test no failures when there are no HAWQ components on a 3-node cluster ... ok
> Test SparkTS is picked when Hive is not installed ... ok
> Test SparkTS is picked when Hive is installed ... ok
> test_noRiskyDictLookups (test_stack_advisor.TestHDP23StackAdvisor) ... ok
>
> Also tested manually.
>
>
> Thanks,
>
> Matt
>
>
Re: Review Request 41641: Recommend PXF component to be collocated
with NAMENODE and DATANODE
Posted by Newton Alex <na...@pivotal.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41641/#review111875
-----------------------------------------------------------
Ship it!
Ship It!
- Newton Alex
On Dec. 22, 2015, 7:36 p.m., Matt wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41641/
> -----------------------------------------------------------
>
> (Updated Dec. 22, 2015, 7:36 p.m.)
>
>
> Review request for Ambari, Alexander Denissov, Alejandro Fernandez, Alexandr Antonenko, bhuvnesh chaudhary, Jaimin Jetly, jun aoki, Jayush Luniya, Lav Jain, Newton Alex, and Oleksandr Diachenko.
>
>
> Bugs: AMBARI-14412
> https://issues.apache.org/jira/browse/AMBARI-14412
>
>
> Repository: ambari
>
>
> Description
> -------
>
> It is recommended that PXF component should be installed on the same hosts which has NAMENODE or/and DATANODE.
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 5ab05fd
> ambari-server/src/test/python/stacks/2.3/common/services-pxf-hdfs.json PRE-CREATION
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py db61f5f
>
> Diff: https://reviews.apache.org/r/41641/diff/
>
>
> Testing
> -------
>
> Added unit test in stack advisor.
> Test passed.
>
> OK
> Running tests for stack:2.3 service:common
> Test that HAWQSTANDBY is not recommended on a single node cluster ... ok
> Test that HAWQSTANDBY is recommended on a 3-node cluster ... ok
> Test no failures when there are no HAWQ components ... ok
> **Test that PXF gets recommended on same host group which has NAMENODE or DATANODE ... ok**
> Test layout validations for HAWQ components on a 3-node cluster ... ok
> Test no failures when there are no HAWQ components on a 3-node cluster ... ok
> Test SparkTS is picked when Hive is not installed ... ok
> Test SparkTS is picked when Hive is installed ... ok
> test_noRiskyDictLookups (test_stack_advisor.TestHDP23StackAdvisor) ... ok
>
> Also tested manually.
>
>
> Thanks,
>
> Matt
>
>
Re: Review Request 41641: Recommend PXF component to be collocated
with NAMENODE and DATANODE
Posted by jun aoki <ju...@gmail.com>.
> On Dec. 28, 2015, 5:53 p.m., Apache Ambari wrote:
> > Please include Srimanth Gunturi in the code review.
@srimanth could you please let us know. We have been waiting for you a few days and will move foward if we don't hear you back.
- jun
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41641/#review112023
-----------------------------------------------------------
On Dec. 28, 2015, 7:42 p.m., Matt wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41641/
> -----------------------------------------------------------
>
> (Updated Dec. 28, 2015, 7:42 p.m.)
>
>
> Review request for Ambari, Alexander Denissov, Alejandro Fernandez, Alexandr Antonenko, bhuvnesh chaudhary, Jaimin Jetly, jun aoki, Jayush Luniya, Lav Jain, Newton Alex, Oleksandr Diachenko, and Srimanth Gunturi.
>
>
> Bugs: AMBARI-14412
> https://issues.apache.org/jira/browse/AMBARI-14412
>
>
> Repository: ambari
>
>
> Description
> -------
>
> It is recommended that PXF component should be installed on the same hosts which has NAMENODE or/and DATANODE.
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 5ab05fd
> ambari-server/src/test/python/stacks/2.3/common/services-pxf-hdfs.json PRE-CREATION
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py db61f5f
>
> Diff: https://reviews.apache.org/r/41641/diff/
>
>
> Testing
> -------
>
> Added unit test in stack advisor.
> Test passed.
>
> OK
> Running tests for stack:2.3 service:common
> Test that HAWQSTANDBY is not recommended on a single node cluster ... ok
> Test that HAWQSTANDBY is recommended on a 3-node cluster ... ok
> Test no failures when there are no HAWQ components ... ok
> **Test that PXF gets recommended on same host group which has NAMENODE or DATANODE ... ok**
> Test layout validations for HAWQ components on a 3-node cluster ... ok
> Test no failures when there are no HAWQ components on a 3-node cluster ... ok
> Test SparkTS is picked when Hive is not installed ... ok
> Test SparkTS is picked when Hive is installed ... ok
> test_noRiskyDictLookups (test_stack_advisor.TestHDP23StackAdvisor) ... ok
>
> Also tested manually.
>
>
> Thanks,
>
> Matt
>
>
Re: Review Request 41641: Recommend PXF component to be collocated
with NAMENODE and DATANODE
Posted by Apache Ambari <ap...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41641/#review112023
-----------------------------------------------------------
Please include Srimanth Gunturi in the code review.
- Apache Ambari
On Dec. 22, 2015, 7:36 p.m., Matt wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41641/
> -----------------------------------------------------------
>
> (Updated Dec. 22, 2015, 7:36 p.m.)
>
>
> Review request for Ambari, Alexander Denissov, Alejandro Fernandez, Alexandr Antonenko, bhuvnesh chaudhary, Jaimin Jetly, jun aoki, Jayush Luniya, Lav Jain, Newton Alex, and Oleksandr Diachenko.
>
>
> Bugs: AMBARI-14412
> https://issues.apache.org/jira/browse/AMBARI-14412
>
>
> Repository: ambari
>
>
> Description
> -------
>
> It is recommended that PXF component should be installed on the same hosts which has NAMENODE or/and DATANODE.
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 5ab05fd
> ambari-server/src/test/python/stacks/2.3/common/services-pxf-hdfs.json PRE-CREATION
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py db61f5f
>
> Diff: https://reviews.apache.org/r/41641/diff/
>
>
> Testing
> -------
>
> Added unit test in stack advisor.
> Test passed.
>
> OK
> Running tests for stack:2.3 service:common
> Test that HAWQSTANDBY is not recommended on a single node cluster ... ok
> Test that HAWQSTANDBY is recommended on a 3-node cluster ... ok
> Test no failures when there are no HAWQ components ... ok
> **Test that PXF gets recommended on same host group which has NAMENODE or DATANODE ... ok**
> Test layout validations for HAWQ components on a 3-node cluster ... ok
> Test no failures when there are no HAWQ components on a 3-node cluster ... ok
> Test SparkTS is picked when Hive is not installed ... ok
> Test SparkTS is picked when Hive is installed ... ok
> test_noRiskyDictLookups (test_stack_advisor.TestHDP23StackAdvisor) ... ok
>
> Also tested manually.
>
>
> Thanks,
>
> Matt
>
>
Re: Review Request 41641: Recommend PXF component to be collocated
with NAMENODE and DATANODE
Posted by Matt <mi...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41641/
-----------------------------------------------------------
(Updated Dec. 22, 2015, 11:36 a.m.)
Review request for Ambari, Alexander Denissov, Alejandro Fernandez, Alexandr Antonenko, bhuvnesh chaudhary, Jaimin Jetly, jun aoki, Jayush Luniya, Lav Jain, Newton Alex, and Oleksandr Diachenko.
Changes
-------
Add unit test results
Bugs: AMBARI-14412
https://issues.apache.org/jira/browse/AMBARI-14412
Repository: ambari
Description
-------
It is recommended that PXF component should be installed on the same hosts which has NAMENODE or/and DATANODE.
Diffs
-----
ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 5ab05fd
ambari-server/src/test/python/stacks/2.3/common/services-pxf-hdfs.json PRE-CREATION
ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py db61f5f
Diff: https://reviews.apache.org/r/41641/diff/
Testing (updated)
-------
Added unit test in stack advisor.
Test passed.
OK
Running tests for stack:2.3 service:common
Test that HAWQSTANDBY is not recommended on a single node cluster ... ok
Test that HAWQSTANDBY is recommended on a 3-node cluster ... ok
Test no failures when there are no HAWQ components ... ok
**Test that PXF gets recommended on same host group which has NAMENODE or DATANODE ... ok**
Test layout validations for HAWQ components on a 3-node cluster ... ok
Test no failures when there are no HAWQ components on a 3-node cluster ... ok
Test SparkTS is picked when Hive is not installed ... ok
Test SparkTS is picked when Hive is installed ... ok
test_noRiskyDictLookups (test_stack_advisor.TestHDP23StackAdvisor) ... ok
Also tested manually.
Thanks,
Matt