You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Lav Jain <lj...@pivotal.io> on 2016/02/25 21:34:53 UTC
Review Request 44038: Set dfs.allow.truncate = True on HDFS during
HAWQ install or add service
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44038/
-----------------------------------------------------------
Review request for Ambari, Alexander Denissov, Alejandro Fernandez, bhuvnesh chaudhary, Goutam Tadi, jun aoki, Jayush Luniya, Matt, Newton Alex, and Oleksandr Diachenko.
Bugs: AMBARI-15189
https://issues.apache.org/jira/browse/AMBARI-15189
Repository: ambari
Description
-------
Setting this param to True is Mandatory, otherwise, failed transactions in HAWQ might generate too many small files and will impact performance.
Diffs
-----
ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 11adca5
ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 690c053
Diff: https://reviews.apache.org/r/44038/diff/
Testing
-------
Unit tests updated with 4 cases.
```
test_createComponentLayoutRecommendations_hawq_1_Host (test_stack_advisor.TestHDP23StackAdvisor)
Test that HAWQSTANDBY is not recommended on a single node cluster ... ok
test_createComponentLayoutRecommendations_hawq_3_Hosts (test_stack_advisor.TestHDP23StackAdvisor)
Test that HAWQSTANDBY is recommended on a 3-node cluster ... ok
test_createComponentLayoutRecommendations_no_hawq_3_Hosts (test_stack_advisor.TestHDP23StackAdvisor)
Test no failures when there are no HAWQ components ... ok
test_createComponentLayoutRecommendations_pxf_co_locate_with_namenode_or_datanode (test_stack_advisor.TestHDP23StackAdvisor)
Test that PXF gets recommended on same host group which has NAMENODE or DATANODE ... ok
test_getComponentLayoutValidations_hawq_3_Hosts (test_stack_advisor.TestHDP23StackAdvisor)
Test layout validations for HAWQ components on a 3-node cluster ... ok
test_getComponentLayoutValidations_nohawq_3_Hosts (test_stack_advisor.TestHDP23StackAdvisor)
Test no failures when there are no HAWQ components on a 3-node cluster ... ok
test_getComponentLayoutValidations_pxf_co_located_with_nn_and_dn (test_stack_advisor.TestHDP23StackAdvisor)
Test NO warning is generated when PXF is co-located with NAMENODE and DATANODE ... ok
test_getComponentLayoutValidations_pxf_not_co_located_with_dn (test_stack_advisor.TestHDP23StackAdvisor)
Test warning is generated when PXF is not co-located with NAMENODE or DATANODE ... ok
test_getComponentLayoutValidations_pxf_not_co_located_with_nn (test_stack_advisor.TestHDP23StackAdvisor)
Test warning is generated when PXF is not co-located with NAMENODE ... ok
test_getComponentLayoutValidations_pxf_not_co_located_with_nn_or_dn (test_stack_advisor.TestHDP23StackAdvisor)
Test warning is generated when PXF is not co-located with NAMENODE or DATANODE ... ok
test_getComponentLayoutValidations_sparkts_no_hive (test_stack_advisor.TestHDP23StackAdvisor)
Test SparkTS is picked when Hive is not installed ... ok
test_getComponentLayoutValidations_sparkts_with_hive (test_stack_advisor.TestHDP23StackAdvisor)
Test SparkTS is picked when Hive is installed ... ok
test_hawqsegmentDatanode (test_stack_advisor.TestHDP23StackAdvisor)
Test that HAWQSegment gets recommended on same host group which has DATANODE ... ok
test_noRiskyDictLookups (test_stack_advisor.TestHDP23StackAdvisor) ... ok
test_recommendHAWQConfigurations (test_stack_advisor.TestHDP23StackAdvisor) ... ok
test_recommendHBASEConfigurations (test_stack_advisor.TestHDP23StackAdvisor) ... ok
test_recommendHDFSConfigurations (test_stack_advisor.TestHDP23StackAdvisor) ... ok
test_recommendHiveConfigurations (test_stack_advisor.TestHDP23StackAdvisor) ... ok
test_recommendKAFKAConfigurations (test_stack_advisor.TestHDP23StackAdvisor) ... ok
test_recommendRangerConfigurations (test_stack_advisor.TestHDP23StackAdvisor) ... ok
test_recommendTezConfigurations (test_stack_advisor.TestHDP23StackAdvisor) ... ok
test_recommendYARNConfigurations (test_stack_advisor.TestHDP23StackAdvisor) ... ok
test_validateHAWQConfigurations (test_stack_advisor.TestHDP23StackAdvisor) ... ok
test_validateHDFSConfigurations (test_stack_advisor.TestHDP23StackAdvisor) ... ok
test_validateHiveConfigurations (test_stack_advisor.TestHDP23StackAdvisor) ... ok
test_validations_hawqsegment (test_stack_advisor.TestHDP23StackAdvisor)
Test validation warning for HAWQ segment not colocated with DATANODE ... ok
```
Thanks,
Lav Jain
Re: Review Request 44038: Set dfs.allow.truncate = True on HDFS
during HAWQ install or add service
Posted by Goutam Tadi <go...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44038/#review120763
-----------------------------------------------------------
Ship it!
Ship It!
- Goutam Tadi
On Feb. 25, 2016, 8:34 p.m., Lav Jain wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44038/
> -----------------------------------------------------------
>
> (Updated Feb. 25, 2016, 8:34 p.m.)
>
>
> Review request for Ambari, Alexander Denissov, Alejandro Fernandez, bhuvnesh chaudhary, Goutam Tadi, jun aoki, Jayush Luniya, Matt, Newton Alex, and Oleksandr Diachenko.
>
>
> Bugs: AMBARI-15189
> https://issues.apache.org/jira/browse/AMBARI-15189
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Setting this param to True is Mandatory, otherwise, failed transactions in HAWQ might generate too many small files and will impact performance.
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 11adca5
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 690c053
>
> Diff: https://reviews.apache.org/r/44038/diff/
>
>
> Testing
> -------
>
> Unit tests updated with 4 cases.
>
> ```
> test_createComponentLayoutRecommendations_hawq_1_Host (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSTANDBY is not recommended on a single node cluster ... ok
> test_createComponentLayoutRecommendations_hawq_3_Hosts (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSTANDBY is recommended on a 3-node cluster ... ok
> test_createComponentLayoutRecommendations_no_hawq_3_Hosts (test_stack_advisor.TestHDP23StackAdvisor)
> Test no failures when there are no HAWQ components ... ok
> test_createComponentLayoutRecommendations_pxf_co_locate_with_namenode_or_datanode (test_stack_advisor.TestHDP23StackAdvisor)
> Test that PXF gets recommended on same host group which has NAMENODE or DATANODE ... ok
> test_getComponentLayoutValidations_hawq_3_Hosts (test_stack_advisor.TestHDP23StackAdvisor)
> Test layout validations for HAWQ components on a 3-node cluster ... ok
> test_getComponentLayoutValidations_nohawq_3_Hosts (test_stack_advisor.TestHDP23StackAdvisor)
> Test no failures when there are no HAWQ components on a 3-node cluster ... ok
> test_getComponentLayoutValidations_pxf_co_located_with_nn_and_dn (test_stack_advisor.TestHDP23StackAdvisor)
> Test NO warning is generated when PXF is co-located with NAMENODE and DATANODE ... ok
> test_getComponentLayoutValidations_pxf_not_co_located_with_dn (test_stack_advisor.TestHDP23StackAdvisor)
> Test warning is generated when PXF is not co-located with NAMENODE or DATANODE ... ok
> test_getComponentLayoutValidations_pxf_not_co_located_with_nn (test_stack_advisor.TestHDP23StackAdvisor)
> Test warning is generated when PXF is not co-located with NAMENODE ... ok
> test_getComponentLayoutValidations_pxf_not_co_located_with_nn_or_dn (test_stack_advisor.TestHDP23StackAdvisor)
> Test warning is generated when PXF is not co-located with NAMENODE or DATANODE ... ok
> test_getComponentLayoutValidations_sparkts_no_hive (test_stack_advisor.TestHDP23StackAdvisor)
> Test SparkTS is picked when Hive is not installed ... ok
> test_getComponentLayoutValidations_sparkts_with_hive (test_stack_advisor.TestHDP23StackAdvisor)
> Test SparkTS is picked when Hive is installed ... ok
> test_hawqsegmentDatanode (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSegment gets recommended on same host group which has DATANODE ... ok
> test_noRiskyDictLookups (test_stack_advisor.TestHDP23StackAdvisor) ... ok
> test_recommendHAWQConfigurations (test_stack_advisor.TestHDP23StackAdvisor) ... ok
> test_recommendHBASEConfigurations (test_stack_advisor.TestHDP23StackAdvisor) ... ok
> test_recommendHDFSConfigurations (test_stack_advisor.TestHDP23StackAdvisor) ... ok
> test_recommendHiveConfigurations (test_stack_advisor.TestHDP23StackAdvisor) ... ok
> test_recommendKAFKAConfigurations (test_stack_advisor.TestHDP23StackAdvisor) ... ok
> test_recommendRangerConfigurations (test_stack_advisor.TestHDP23StackAdvisor) ... ok
> test_recommendTezConfigurations (test_stack_advisor.TestHDP23StackAdvisor) ... ok
> test_recommendYARNConfigurations (test_stack_advisor.TestHDP23StackAdvisor) ... ok
> test_validateHAWQConfigurations (test_stack_advisor.TestHDP23StackAdvisor) ... ok
> test_validateHDFSConfigurations (test_stack_advisor.TestHDP23StackAdvisor) ... ok
> test_validateHiveConfigurations (test_stack_advisor.TestHDP23StackAdvisor) ... ok
> test_validations_hawqsegment (test_stack_advisor.TestHDP23StackAdvisor)
> Test validation warning for HAWQ segment not colocated with DATANODE ... ok
> ```
>
>
> Thanks,
>
> Lav Jain
>
>