You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Dmytro Sen <ds...@hortonworks.com> on 2015/12/30 14:04:58 UTC
Review Request 41797: HDFS Recommendation: dfs.datanode.du.reserved
should be set to 10%-15% of volume size
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41797/
-----------------------------------------------------------
Review request for Ambari, Andrew Onischuk and Vitalyi Brodetskyi.
Bugs: AMBARI-14524
https://issues.apache.org/jira/browse/AMBARI-14524
Repository: ambari
Description
-------
PROBLEM: Ambari should give recommendations on this setting: dfs.datanode.du.reserved
The default is 1GB and that is too low for a lot of our production customers. If this is too low and the space on the datanode fills up, the nodemanagers will start failing the health checks and we get into a bad state with not enough nodemanagers being up. At this point the admin tries to run the balancer and hope that it will balance very quickly.
There should be a recommendation on this value based on the disk space available in the datanode volume. In addition, if the user sets it too low there should be a warning saying this will cause problems if enough buffer is not kept.
Diffs
-----
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py a723f75
ambari-server/src/main/resources/stacks/HDP/2.2/services/stack_advisor.py 12f85b8
ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 5ab05fd
ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py d79b2ad
ambari-server/src/test/python/stacks/2.1/common/test_stack_advisor.py 63d34af
ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py 1ee3e7a
ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py db61f5f
Diff: https://reviews.apache.org/r/41797/diff/
Testing
-------
Ran 251 tests in 8.189s
OK
----------------------------------------------------------------------
Total run:858
Total errors:0
Total failures:0
OK
Thanks,
Dmytro Sen
Re: Review Request 41797: HDFS Recommendation:
dfs.datanode.du.reserved should be set to 10%-15% of volume size
Posted by Andrew Onischuk <ao...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41797/#review112314
-----------------------------------------------------------
Ship it!
Ship It!
- Andrew Onischuk
On Dec. 30, 2015, 2:30 p.m., Dmytro Sen wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41797/
> -----------------------------------------------------------
>
> (Updated Dec. 30, 2015, 2:30 p.m.)
>
>
> Review request for Ambari, Andrew Onischuk and Vitalyi Brodetskyi.
>
>
> Bugs: AMBARI-14524
> https://issues.apache.org/jira/browse/AMBARI-14524
>
>
> Repository: ambari
>
>
> Description
> -------
>
> PROBLEM: Ambari should give recommendations on this setting: dfs.datanode.du.reserved
> The default is 1GB and that is too low for a lot of our production customers. If this is too low and the space on the datanode fills up, the nodemanagers will start failing the health checks and we get into a bad state with not enough nodemanagers being up. At this point the admin tries to run the balancer and hope that it will balance very quickly.
> There should be a recommendation on this value based on the disk space available in the datanode volume. In addition, if the user sets it too low there should be a warning saying this will cause problems if enough buffer is not kept.
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/configuration/hdfs-site.xml 3282522
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 7e2871b
> ambari-server/src/main/resources/stacks/HDP/2.2/services/stack_advisor.py 12f85b8
> ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 5ab05fd
> ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py 6699e94
> ambari-server/src/test/python/stacks/2.1/common/test_stack_advisor.py 63d34af
> ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py 4e5458e
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py db61f5f
>
> Diff: https://reviews.apache.org/r/41797/diff/
>
>
> Testing
> -------
>
> Ran 251 tests in 8.189s
>
> OK
> ----------------------------------------------------------------------
> Total run:858
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Dmytro Sen
>
>
Re: Review Request 41797: HDFS Recommendation:
dfs.datanode.du.reserved should be set to 10%-15% of volume size
Posted by Vitalyi Brodetskyi <vb...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41797/#review112315
-----------------------------------------------------------
Ship it!
Ship It!
- Vitalyi Brodetskyi
On Гру. 30, 2015, 2:30 після полудня, Dmytro Sen wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41797/
> -----------------------------------------------------------
>
> (Updated Гру. 30, 2015, 2:30 після полудня)
>
>
> Review request for Ambari, Andrew Onischuk and Vitalyi Brodetskyi.
>
>
> Bugs: AMBARI-14524
> https://issues.apache.org/jira/browse/AMBARI-14524
>
>
> Repository: ambari
>
>
> Description
> -------
>
> PROBLEM: Ambari should give recommendations on this setting: dfs.datanode.du.reserved
> The default is 1GB and that is too low for a lot of our production customers. If this is too low and the space on the datanode fills up, the nodemanagers will start failing the health checks and we get into a bad state with not enough nodemanagers being up. At this point the admin tries to run the balancer and hope that it will balance very quickly.
> There should be a recommendation on this value based on the disk space available in the datanode volume. In addition, if the user sets it too low there should be a warning saying this will cause problems if enough buffer is not kept.
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/configuration/hdfs-site.xml 3282522
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 7e2871b
> ambari-server/src/main/resources/stacks/HDP/2.2/services/stack_advisor.py 12f85b8
> ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 5ab05fd
> ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py 6699e94
> ambari-server/src/test/python/stacks/2.1/common/test_stack_advisor.py 63d34af
> ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py 4e5458e
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py db61f5f
>
> Diff: https://reviews.apache.org/r/41797/diff/
>
>
> Testing
> -------
>
> Ran 251 tests in 8.189s
>
> OK
> ----------------------------------------------------------------------
> Total run:858
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Dmytro Sen
>
>
Re: Review Request 41797: HDFS Recommendation:
dfs.datanode.du.reserved should be set to 10%-15% of volume size
Posted by Dmytro Sen <ds...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41797/
-----------------------------------------------------------
(Updated Дек. 30, 2015, 2:30 п.п.)
Review request for Ambari, Andrew Onischuk and Vitalyi Brodetskyi.
Bugs: AMBARI-14524
https://issues.apache.org/jira/browse/AMBARI-14524
Repository: ambari
Description
-------
PROBLEM: Ambari should give recommendations on this setting: dfs.datanode.du.reserved
The default is 1GB and that is too low for a lot of our production customers. If this is too low and the space on the datanode fills up, the nodemanagers will start failing the health checks and we get into a bad state with not enough nodemanagers being up. At this point the admin tries to run the balancer and hope that it will balance very quickly.
There should be a recommendation on this value based on the disk space available in the datanode volume. In addition, if the user sets it too low there should be a warning saying this will cause problems if enough buffer is not kept.
Diffs (updated)
-----
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/configuration/hdfs-site.xml 3282522
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 7e2871b
ambari-server/src/main/resources/stacks/HDP/2.2/services/stack_advisor.py 12f85b8
ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 5ab05fd
ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py 6699e94
ambari-server/src/test/python/stacks/2.1/common/test_stack_advisor.py 63d34af
ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py 4e5458e
ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py db61f5f
Diff: https://reviews.apache.org/r/41797/diff/
Testing
-------
Ran 251 tests in 8.189s
OK
----------------------------------------------------------------------
Total run:858
Total errors:0
Total failures:0
OK
Thanks,
Dmytro Sen