You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Colin Patrick McCabe (JIRA)" <ji...@apache.org> on 2012/10/08 23:20:03 UTC
[jira] [Created] (HDFS-4021) Misleading error message when
resources are low on the NameNode
Colin Patrick McCabe created HDFS-4021:
------------------------------------------
Summary: Misleading error message when resources are low on the NameNode
Key: HDFS-4021
URL: https://issues.apache.org/jira/browse/HDFS-4021
Project: Hadoop HDFS
Issue Type: Bug
Components: name-node
Affects Versions: 2.0.3-alpha
Reporter: Colin Patrick McCabe
Priority: Minor
When resources are low on the namenode, it enters SafeMode with a message like this:
{code}
14:00:10,666 WARN NameNodeResourceChecker:89 - Space available on
volume '/dev/hda1' is 104357888, which is below the configured
reserved amount 104857600
14:00:10,670 WARN FSNamesystem:3190 - NameNode low on available disk
space. Entering safe mode.
14:00:10,670 INFO StateChange:3836 - STATE* Safe mode is ON.
Resources are low on NN. Safe mode must be turned off manually.
{code}
However, turning off safe mode manually has no effect, since it immediately puts itself into Safe Mode again with a log message like this:
{code}
14:00:10,666 WARN NameNodeResourceChecker:89 - Space available on
volume '/dev/hda1' is 104357888, which is below the configured
reserved amount 104857600
14:00:10,670 WARN FSNamesystem:3190 - NameNode low on available disk
space. Entering safe mode.
14:00:10,670 INFO StateChange:3836 - STATE* Safe mode is ON.
Resources are low on NN. Safe mode must be turned off manually.
{code}
>From the shell, it looks like this:
{code}
[cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode get
Safe mode is ON
[cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode leave
Safe mode is OFF
[cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode get
Safe mode is ON
{code}
It seems like we should change the message about turning off safe mode manually, if turning off safe mode manually does not actually work for the case where resources are low. Probably we need to explain that safe mode should be turned off manually *after* adding more resources. As it is, the error message seems misleading.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4021) Misleading error message when
resources are low on the NameNode
Posted by "Aaron T. Myers (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron T. Myers updated HDFS-4021:
---------------------------------
Resolution: Fixed
Fix Version/s: 2.0.3-alpha
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
I've just committed this to trunk and branch-2.
Thanks a lot for the contribution, Christopher.
> Misleading error message when resources are low on the NameNode
> ---------------------------------------------------------------
>
> Key: HDFS-4021
> URL: https://issues.apache.org/jira/browse/HDFS-4021
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Christopher Conner
> Priority: Minor
> Labels: newbie
> Fix For: 2.0.3-alpha
>
> Attachments: HDFS-4021.patch, HDFS-4021.patch
>
>
> When resources are low on the namenode, it enters SafeMode with a message like this:
> {code}
> 14:00:10,666 WARN NameNodeResourceChecker:89 - Space available on
> volume '/dev/hda1' is 104357888, which is below the configured
> reserved amount 104857600
> 14:00:10,670 WARN FSNamesystem:3190 - NameNode low on available disk
> space. Entering safe mode.
> 14:00:10,670 INFO StateChange:3836 - STATE* Safe mode is ON.
> Resources are low on NN. Safe mode must be turned off manually.
> {code}
> However, turning off safe mode manually has no effect, since it immediately puts itself into Safe Mode again with a log message like this:
> {code}
> 14:00:10,666 WARN NameNodeResourceChecker:89 - Space available on
> volume '/dev/hda1' is 104357888, which is below the configured
> reserved amount 104857600
> 14:00:10,670 WARN FSNamesystem:3190 - NameNode low on available disk
> space. Entering safe mode.
> 14:00:10,670 INFO StateChange:3836 - STATE* Safe mode is ON.
> Resources are low on NN. Safe mode must be turned off manually.
> {code}
> From the shell, it looks like this:
> {code}
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode get
> Safe mode is ON
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode leave
> Safe mode is OFF
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode get
> Safe mode is ON
> {code}
> It seems like we should change the message about turning off safe mode manually, if turning off safe mode manually does not actually work for the case where resources are low. Probably we need to explain that safe mode should be turned off manually *after* adding more resources. As it is, the error message seems misleading.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4021) Misleading error message when
resources are low on the NameNode
Posted by "Christopher Conner (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Christopher Conner updated HDFS-4021:
-------------------------------------
Attachment: HDFS-4021.patch
> Misleading error message when resources are low on the NameNode
> ---------------------------------------------------------------
>
> Key: HDFS-4021
> URL: https://issues.apache.org/jira/browse/HDFS-4021
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Priority: Minor
> Labels: newbie
> Attachments: HDFS-4021.patch
>
>
> When resources are low on the namenode, it enters SafeMode with a message like this:
> {code}
> 14:00:10,666 WARN NameNodeResourceChecker:89 - Space available on
> volume '/dev/hda1' is 104357888, which is below the configured
> reserved amount 104857600
> 14:00:10,670 WARN FSNamesystem:3190 - NameNode low on available disk
> space. Entering safe mode.
> 14:00:10,670 INFO StateChange:3836 - STATE* Safe mode is ON.
> Resources are low on NN. Safe mode must be turned off manually.
> {code}
> However, turning off safe mode manually has no effect, since it immediately puts itself into Safe Mode again with a log message like this:
> {code}
> 14:00:10,666 WARN NameNodeResourceChecker:89 - Space available on
> volume '/dev/hda1' is 104357888, which is below the configured
> reserved amount 104857600
> 14:00:10,670 WARN FSNamesystem:3190 - NameNode low on available disk
> space. Entering safe mode.
> 14:00:10,670 INFO StateChange:3836 - STATE* Safe mode is ON.
> Resources are low on NN. Safe mode must be turned off manually.
> {code}
> From the shell, it looks like this:
> {code}
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode get
> Safe mode is ON
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode leave
> Safe mode is OFF
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode get
> Safe mode is ON
> {code}
> It seems like we should change the message about turning off safe mode manually, if turning off safe mode manually does not actually work for the case where resources are low. Probably we need to explain that safe mode should be turned off manually *after* adding more resources. As it is, the error message seems misleading.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4021) Misleading error message when
resources are low on the NameNode
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13473735#comment-13473735 ]
Hadoop QA commented on HDFS-4021:
---------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12548629/HDFS-4021.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/3302//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3302//console
This message is automatically generated.
> Misleading error message when resources are low on the NameNode
> ---------------------------------------------------------------
>
> Key: HDFS-4021
> URL: https://issues.apache.org/jira/browse/HDFS-4021
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Christopher Conner
> Priority: Minor
> Labels: newbie
> Attachments: HDFS-4021.patch, HDFS-4021.patch
>
>
> When resources are low on the namenode, it enters SafeMode with a message like this:
> {code}
> 14:00:10,666 WARN NameNodeResourceChecker:89 - Space available on
> volume '/dev/hda1' is 104357888, which is below the configured
> reserved amount 104857600
> 14:00:10,670 WARN FSNamesystem:3190 - NameNode low on available disk
> space. Entering safe mode.
> 14:00:10,670 INFO StateChange:3836 - STATE* Safe mode is ON.
> Resources are low on NN. Safe mode must be turned off manually.
> {code}
> However, turning off safe mode manually has no effect, since it immediately puts itself into Safe Mode again with a log message like this:
> {code}
> 14:00:10,666 WARN NameNodeResourceChecker:89 - Space available on
> volume '/dev/hda1' is 104357888, which is below the configured
> reserved amount 104857600
> 14:00:10,670 WARN FSNamesystem:3190 - NameNode low on available disk
> space. Entering safe mode.
> 14:00:10,670 INFO StateChange:3836 - STATE* Safe mode is ON.
> Resources are low on NN. Safe mode must be turned off manually.
> {code}
> From the shell, it looks like this:
> {code}
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode get
> Safe mode is ON
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode leave
> Safe mode is OFF
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode get
> Safe mode is ON
> {code}
> It seems like we should change the message about turning off safe mode manually, if turning off safe mode manually does not actually work for the case where resources are low. Probably we need to explain that safe mode should be turned off manually *after* adding more resources. As it is, the error message seems misleading.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4021) Misleading error message when
resources are low on the NameNode
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13473766#comment-13473766 ]
Hudson commented on HDFS-4021:
------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #2866 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2866/])
HDFS-4021. Misleading error message when resources are low on the NameNode. Contributed by Christopher Conner. (Revision 1396881)
Result = FAILURE
atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1396881
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
> Misleading error message when resources are low on the NameNode
> ---------------------------------------------------------------
>
> Key: HDFS-4021
> URL: https://issues.apache.org/jira/browse/HDFS-4021
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Christopher Conner
> Priority: Minor
> Labels: newbie
> Fix For: 2.0.3-alpha
>
> Attachments: HDFS-4021.patch, HDFS-4021.patch
>
>
> When resources are low on the namenode, it enters SafeMode with a message like this:
> {code}
> 14:00:10,666 WARN NameNodeResourceChecker:89 - Space available on
> volume '/dev/hda1' is 104357888, which is below the configured
> reserved amount 104857600
> 14:00:10,670 WARN FSNamesystem:3190 - NameNode low on available disk
> space. Entering safe mode.
> 14:00:10,670 INFO StateChange:3836 - STATE* Safe mode is ON.
> Resources are low on NN. Safe mode must be turned off manually.
> {code}
> However, turning off safe mode manually has no effect, since it immediately puts itself into Safe Mode again with a log message like this:
> {code}
> 14:00:10,666 WARN NameNodeResourceChecker:89 - Space available on
> volume '/dev/hda1' is 104357888, which is below the configured
> reserved amount 104857600
> 14:00:10,670 WARN FSNamesystem:3190 - NameNode low on available disk
> space. Entering safe mode.
> 14:00:10,670 INFO StateChange:3836 - STATE* Safe mode is ON.
> Resources are low on NN. Safe mode must be turned off manually.
> {code}
> From the shell, it looks like this:
> {code}
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode get
> Safe mode is ON
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode leave
> Safe mode is OFF
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode get
> Safe mode is ON
> {code}
> It seems like we should change the message about turning off safe mode manually, if turning off safe mode manually does not actually work for the case where resources are low. Probably we need to explain that safe mode should be turned off manually *after* adding more resources. As it is, the error message seems misleading.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HDFS-4021) Misleading error message when
resources are low on the NameNode
Posted by "Aaron T. Myers (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron T. Myers reassigned HDFS-4021:
------------------------------------
Assignee: Christopher Conner (was: Colin Patrick McCabe)
> Misleading error message when resources are low on the NameNode
> ---------------------------------------------------------------
>
> Key: HDFS-4021
> URL: https://issues.apache.org/jira/browse/HDFS-4021
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Christopher Conner
> Priority: Minor
> Labels: newbie
> Attachments: HDFS-4021.patch
>
>
> When resources are low on the namenode, it enters SafeMode with a message like this:
> {code}
> 14:00:10,666 WARN NameNodeResourceChecker:89 - Space available on
> volume '/dev/hda1' is 104357888, which is below the configured
> reserved amount 104857600
> 14:00:10,670 WARN FSNamesystem:3190 - NameNode low on available disk
> space. Entering safe mode.
> 14:00:10,670 INFO StateChange:3836 - STATE* Safe mode is ON.
> Resources are low on NN. Safe mode must be turned off manually.
> {code}
> However, turning off safe mode manually has no effect, since it immediately puts itself into Safe Mode again with a log message like this:
> {code}
> 14:00:10,666 WARN NameNodeResourceChecker:89 - Space available on
> volume '/dev/hda1' is 104357888, which is below the configured
> reserved amount 104857600
> 14:00:10,670 WARN FSNamesystem:3190 - NameNode low on available disk
> space. Entering safe mode.
> 14:00:10,670 INFO StateChange:3836 - STATE* Safe mode is ON.
> Resources are low on NN. Safe mode must be turned off manually.
> {code}
> From the shell, it looks like this:
> {code}
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode get
> Safe mode is ON
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode leave
> Safe mode is OFF
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode get
> Safe mode is ON
> {code}
> It seems like we should change the message about turning off safe mode manually, if turning off safe mode manually does not actually work for the case where resources are low. Probably we need to explain that safe mode should be turned off manually *after* adding more resources. As it is, the error message seems misleading.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4021) Misleading error message when
resources are low on the NameNode
Posted by "Christopher Conner (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Christopher Conner updated HDFS-4021:
-------------------------------------
Attachment: (was: HDFS-4021.patch)
> Misleading error message when resources are low on the NameNode
> ---------------------------------------------------------------
>
> Key: HDFS-4021
> URL: https://issues.apache.org/jira/browse/HDFS-4021
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Priority: Minor
> Labels: newbie
> Attachments: HDFS-4021.patch
>
>
> When resources are low on the namenode, it enters SafeMode with a message like this:
> {code}
> 14:00:10,666 WARN NameNodeResourceChecker:89 - Space available on
> volume '/dev/hda1' is 104357888, which is below the configured
> reserved amount 104857600
> 14:00:10,670 WARN FSNamesystem:3190 - NameNode low on available disk
> space. Entering safe mode.
> 14:00:10,670 INFO StateChange:3836 - STATE* Safe mode is ON.
> Resources are low on NN. Safe mode must be turned off manually.
> {code}
> However, turning off safe mode manually has no effect, since it immediately puts itself into Safe Mode again with a log message like this:
> {code}
> 14:00:10,666 WARN NameNodeResourceChecker:89 - Space available on
> volume '/dev/hda1' is 104357888, which is below the configured
> reserved amount 104857600
> 14:00:10,670 WARN FSNamesystem:3190 - NameNode low on available disk
> space. Entering safe mode.
> 14:00:10,670 INFO StateChange:3836 - STATE* Safe mode is ON.
> Resources are low on NN. Safe mode must be turned off manually.
> {code}
> From the shell, it looks like this:
> {code}
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode get
> Safe mode is ON
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode leave
> Safe mode is OFF
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode get
> Safe mode is ON
> {code}
> It seems like we should change the message about turning off safe mode manually, if turning off safe mode manually does not actually work for the case where resources are low. Probably we need to explain that safe mode should be turned off manually *after* adding more resources. As it is, the error message seems misleading.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4021) Misleading error message when
resources are low on the NameNode
Posted by "Aaron T. Myers (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13473536#comment-13473536 ]
Aaron T. Myers commented on HDFS-4021:
--------------------------------------
Patch looks pretty good to me. One suggestion: instead of "Please _add_ more resources" I recommend we change the message text to "Please _add or free_ more resources." Specifically, this message usually indicates that the NN is low on local disk space, so it's unlikely they'll actually be able to hot swap in _more_ resources, and instead will have to delete some stuff. Do folks agree?
> Misleading error message when resources are low on the NameNode
> ---------------------------------------------------------------
>
> Key: HDFS-4021
> URL: https://issues.apache.org/jira/browse/HDFS-4021
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Christopher Conner
> Priority: Minor
> Labels: newbie
> Attachments: HDFS-4021.patch
>
>
> When resources are low on the namenode, it enters SafeMode with a message like this:
> {code}
> 14:00:10,666 WARN NameNodeResourceChecker:89 - Space available on
> volume '/dev/hda1' is 104357888, which is below the configured
> reserved amount 104857600
> 14:00:10,670 WARN FSNamesystem:3190 - NameNode low on available disk
> space. Entering safe mode.
> 14:00:10,670 INFO StateChange:3836 - STATE* Safe mode is ON.
> Resources are low on NN. Safe mode must be turned off manually.
> {code}
> However, turning off safe mode manually has no effect, since it immediately puts itself into Safe Mode again with a log message like this:
> {code}
> 14:00:10,666 WARN NameNodeResourceChecker:89 - Space available on
> volume '/dev/hda1' is 104357888, which is below the configured
> reserved amount 104857600
> 14:00:10,670 WARN FSNamesystem:3190 - NameNode low on available disk
> space. Entering safe mode.
> 14:00:10,670 INFO StateChange:3836 - STATE* Safe mode is ON.
> Resources are low on NN. Safe mode must be turned off manually.
> {code}
> From the shell, it looks like this:
> {code}
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode get
> Safe mode is ON
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode leave
> Safe mode is OFF
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode get
> Safe mode is ON
> {code}
> It seems like we should change the message about turning off safe mode manually, if turning off safe mode manually does not actually work for the case where resources are low. Probably we need to explain that safe mode should be turned off manually *after* adding more resources. As it is, the error message seems misleading.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4021) Misleading error message when
resources are low on the NameNode
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13473809#comment-13473809 ]
Hudson commented on HDFS-4021:
------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #2905 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2905/])
HDFS-4021. Misleading error message when resources are low on the NameNode. Contributed by Christopher Conner. (Revision 1396881)
Result = SUCCESS
atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1396881
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
> Misleading error message when resources are low on the NameNode
> ---------------------------------------------------------------
>
> Key: HDFS-4021
> URL: https://issues.apache.org/jira/browse/HDFS-4021
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Christopher Conner
> Priority: Minor
> Labels: newbie
> Fix For: 2.0.3-alpha
>
> Attachments: HDFS-4021.patch, HDFS-4021.patch
>
>
> When resources are low on the namenode, it enters SafeMode with a message like this:
> {code}
> 14:00:10,666 WARN NameNodeResourceChecker:89 - Space available on
> volume '/dev/hda1' is 104357888, which is below the configured
> reserved amount 104857600
> 14:00:10,670 WARN FSNamesystem:3190 - NameNode low on available disk
> space. Entering safe mode.
> 14:00:10,670 INFO StateChange:3836 - STATE* Safe mode is ON.
> Resources are low on NN. Safe mode must be turned off manually.
> {code}
> However, turning off safe mode manually has no effect, since it immediately puts itself into Safe Mode again with a log message like this:
> {code}
> 14:00:10,666 WARN NameNodeResourceChecker:89 - Space available on
> volume '/dev/hda1' is 104357888, which is below the configured
> reserved amount 104857600
> 14:00:10,670 WARN FSNamesystem:3190 - NameNode low on available disk
> space. Entering safe mode.
> 14:00:10,670 INFO StateChange:3836 - STATE* Safe mode is ON.
> Resources are low on NN. Safe mode must be turned off manually.
> {code}
> From the shell, it looks like this:
> {code}
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode get
> Safe mode is ON
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode leave
> Safe mode is OFF
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode get
> Safe mode is ON
> {code}
> It seems like we should change the message about turning off safe mode manually, if turning off safe mode manually does not actually work for the case where resources are low. Probably we need to explain that safe mode should be turned off manually *after* adding more resources. As it is, the error message seems misleading.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4021) Misleading error message when
resources are low on the NameNode
Posted by "Christopher Conner (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Christopher Conner updated HDFS-4021:
-------------------------------------
Status: Patch Available (was: Open)
Changed warning message from:
Resources are low on NN. Safe mode must be turned off manually.
To:
Resources are low on NN. Please add more resources then turn off safe mode manually. NOTE: If you turn off safe mode before adding resources, the NN will immediately return to safe mode.
> Misleading error message when resources are low on the NameNode
> ---------------------------------------------------------------
>
> Key: HDFS-4021
> URL: https://issues.apache.org/jira/browse/HDFS-4021
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Priority: Minor
> Labels: newbie
>
> When resources are low on the namenode, it enters SafeMode with a message like this:
> {code}
> 14:00:10,666 WARN NameNodeResourceChecker:89 - Space available on
> volume '/dev/hda1' is 104357888, which is below the configured
> reserved amount 104857600
> 14:00:10,670 WARN FSNamesystem:3190 - NameNode low on available disk
> space. Entering safe mode.
> 14:00:10,670 INFO StateChange:3836 - STATE* Safe mode is ON.
> Resources are low on NN. Safe mode must be turned off manually.
> {code}
> However, turning off safe mode manually has no effect, since it immediately puts itself into Safe Mode again with a log message like this:
> {code}
> 14:00:10,666 WARN NameNodeResourceChecker:89 - Space available on
> volume '/dev/hda1' is 104357888, which is below the configured
> reserved amount 104857600
> 14:00:10,670 WARN FSNamesystem:3190 - NameNode low on available disk
> space. Entering safe mode.
> 14:00:10,670 INFO StateChange:3836 - STATE* Safe mode is ON.
> Resources are low on NN. Safe mode must be turned off manually.
> {code}
> From the shell, it looks like this:
> {code}
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode get
> Safe mode is ON
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode leave
> Safe mode is OFF
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode get
> Safe mode is ON
> {code}
> It seems like we should change the message about turning off safe mode manually, if turning off safe mode manually does not actually work for the case where resources are low. Probably we need to explain that safe mode should be turned off manually *after* adding more resources. As it is, the error message seems misleading.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4021) Misleading error message when
resources are low on the NameNode
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13474139#comment-13474139 ]
Hudson commented on HDFS-4021:
------------------------------
Integrated in Hadoop-Mapreduce-trunk #1223 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1223/])
HDFS-4021. Misleading error message when resources are low on the NameNode. Contributed by Christopher Conner. (Revision 1396881)
Result = FAILURE
atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1396881
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
> Misleading error message when resources are low on the NameNode
> ---------------------------------------------------------------
>
> Key: HDFS-4021
> URL: https://issues.apache.org/jira/browse/HDFS-4021
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Christopher Conner
> Priority: Minor
> Labels: newbie
> Fix For: 2.0.3-alpha
>
> Attachments: HDFS-4021.patch, HDFS-4021.patch
>
>
> When resources are low on the namenode, it enters SafeMode with a message like this:
> {code}
> 14:00:10,666 WARN NameNodeResourceChecker:89 - Space available on
> volume '/dev/hda1' is 104357888, which is below the configured
> reserved amount 104857600
> 14:00:10,670 WARN FSNamesystem:3190 - NameNode low on available disk
> space. Entering safe mode.
> 14:00:10,670 INFO StateChange:3836 - STATE* Safe mode is ON.
> Resources are low on NN. Safe mode must be turned off manually.
> {code}
> However, turning off safe mode manually has no effect, since it immediately puts itself into Safe Mode again with a log message like this:
> {code}
> 14:00:10,666 WARN NameNodeResourceChecker:89 - Space available on
> volume '/dev/hda1' is 104357888, which is below the configured
> reserved amount 104857600
> 14:00:10,670 WARN FSNamesystem:3190 - NameNode low on available disk
> space. Entering safe mode.
> 14:00:10,670 INFO StateChange:3836 - STATE* Safe mode is ON.
> Resources are low on NN. Safe mode must be turned off manually.
> {code}
> From the shell, it looks like this:
> {code}
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode get
> Safe mode is ON
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode leave
> Safe mode is OFF
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode get
> Safe mode is ON
> {code}
> It seems like we should change the message about turning off safe mode manually, if turning off safe mode manually does not actually work for the case where resources are low. Probably we need to explain that safe mode should be turned off manually *after* adding more resources. As it is, the error message seems misleading.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4021) Misleading error message when
resources are low on the NameNode
Posted by "Christopher Conner (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Christopher Conner updated HDFS-4021:
-------------------------------------
Attachment: HDFS-4021.patch
New patch implementing Aaron's recommended changes.
> Misleading error message when resources are low on the NameNode
> ---------------------------------------------------------------
>
> Key: HDFS-4021
> URL: https://issues.apache.org/jira/browse/HDFS-4021
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Christopher Conner
> Priority: Minor
> Labels: newbie
> Attachments: HDFS-4021.patch, HDFS-4021.patch
>
>
> When resources are low on the namenode, it enters SafeMode with a message like this:
> {code}
> 14:00:10,666 WARN NameNodeResourceChecker:89 - Space available on
> volume '/dev/hda1' is 104357888, which is below the configured
> reserved amount 104857600
> 14:00:10,670 WARN FSNamesystem:3190 - NameNode low on available disk
> space. Entering safe mode.
> 14:00:10,670 INFO StateChange:3836 - STATE* Safe mode is ON.
> Resources are low on NN. Safe mode must be turned off manually.
> {code}
> However, turning off safe mode manually has no effect, since it immediately puts itself into Safe Mode again with a log message like this:
> {code}
> 14:00:10,666 WARN NameNodeResourceChecker:89 - Space available on
> volume '/dev/hda1' is 104357888, which is below the configured
> reserved amount 104857600
> 14:00:10,670 WARN FSNamesystem:3190 - NameNode low on available disk
> space. Entering safe mode.
> 14:00:10,670 INFO StateChange:3836 - STATE* Safe mode is ON.
> Resources are low on NN. Safe mode must be turned off manually.
> {code}
> From the shell, it looks like this:
> {code}
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode get
> Safe mode is ON
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode leave
> Safe mode is OFF
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode get
> Safe mode is ON
> {code}
> It seems like we should change the message about turning off safe mode manually, if turning off safe mode manually does not actually work for the case where resources are low. Probably we need to explain that safe mode should be turned off manually *after* adding more resources. As it is, the error message seems misleading.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4021) Misleading error message when
resources are low on the NameNode
Posted by "Colin Patrick McCabe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472904#comment-13472904 ]
Colin Patrick McCabe commented on HDFS-4021:
--------------------------------------------
Thanks for reassigning this, atm. I fat-fingered the "assign to me" button.
> Misleading error message when resources are low on the NameNode
> ---------------------------------------------------------------
>
> Key: HDFS-4021
> URL: https://issues.apache.org/jira/browse/HDFS-4021
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Christopher Conner
> Priority: Minor
> Labels: newbie
> Attachments: HDFS-4021.patch
>
>
> When resources are low on the namenode, it enters SafeMode with a message like this:
> {code}
> 14:00:10,666 WARN NameNodeResourceChecker:89 - Space available on
> volume '/dev/hda1' is 104357888, which is below the configured
> reserved amount 104857600
> 14:00:10,670 WARN FSNamesystem:3190 - NameNode low on available disk
> space. Entering safe mode.
> 14:00:10,670 INFO StateChange:3836 - STATE* Safe mode is ON.
> Resources are low on NN. Safe mode must be turned off manually.
> {code}
> However, turning off safe mode manually has no effect, since it immediately puts itself into Safe Mode again with a log message like this:
> {code}
> 14:00:10,666 WARN NameNodeResourceChecker:89 - Space available on
> volume '/dev/hda1' is 104357888, which is below the configured
> reserved amount 104857600
> 14:00:10,670 WARN FSNamesystem:3190 - NameNode low on available disk
> space. Entering safe mode.
> 14:00:10,670 INFO StateChange:3836 - STATE* Safe mode is ON.
> Resources are low on NN. Safe mode must be turned off manually.
> {code}
> From the shell, it looks like this:
> {code}
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode get
> Safe mode is ON
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode leave
> Safe mode is OFF
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode get
> Safe mode is ON
> {code}
> It seems like we should change the message about turning off safe mode manually, if turning off safe mode manually does not actually work for the case where resources are low. Probably we need to explain that safe mode should be turned off manually *after* adding more resources. As it is, the error message seems misleading.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4021) Misleading error message when
resources are low on the NameNode
Posted by "Colin Patrick McCabe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472836#comment-13472836 ]
Colin Patrick McCabe commented on HDFS-4021:
--------------------------------------------
Looks good to me.
> Misleading error message when resources are low on the NameNode
> ---------------------------------------------------------------
>
> Key: HDFS-4021
> URL: https://issues.apache.org/jira/browse/HDFS-4021
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Priority: Minor
> Labels: newbie
> Attachments: HDFS-4021.patch
>
>
> When resources are low on the namenode, it enters SafeMode with a message like this:
> {code}
> 14:00:10,666 WARN NameNodeResourceChecker:89 - Space available on
> volume '/dev/hda1' is 104357888, which is below the configured
> reserved amount 104857600
> 14:00:10,670 WARN FSNamesystem:3190 - NameNode low on available disk
> space. Entering safe mode.
> 14:00:10,670 INFO StateChange:3836 - STATE* Safe mode is ON.
> Resources are low on NN. Safe mode must be turned off manually.
> {code}
> However, turning off safe mode manually has no effect, since it immediately puts itself into Safe Mode again with a log message like this:
> {code}
> 14:00:10,666 WARN NameNodeResourceChecker:89 - Space available on
> volume '/dev/hda1' is 104357888, which is below the configured
> reserved amount 104857600
> 14:00:10,670 WARN FSNamesystem:3190 - NameNode low on available disk
> space. Entering safe mode.
> 14:00:10,670 INFO StateChange:3836 - STATE* Safe mode is ON.
> Resources are low on NN. Safe mode must be turned off manually.
> {code}
> From the shell, it looks like this:
> {code}
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode get
> Safe mode is ON
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode leave
> Safe mode is OFF
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode get
> Safe mode is ON
> {code}
> It seems like we should change the message about turning off safe mode manually, if turning off safe mode manually does not actually work for the case where resources are low. Probably we need to explain that safe mode should be turned off manually *after* adding more resources. As it is, the error message seems misleading.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4021) Misleading error message when
resources are low on the NameNode
Posted by "Aaron T. Myers (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13473548#comment-13473548 ]
Aaron T. Myers commented on HDFS-4021:
--------------------------------------
+1 pending Jenkins.
> Misleading error message when resources are low on the NameNode
> ---------------------------------------------------------------
>
> Key: HDFS-4021
> URL: https://issues.apache.org/jira/browse/HDFS-4021
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Christopher Conner
> Priority: Minor
> Labels: newbie
> Attachments: HDFS-4021.patch, HDFS-4021.patch
>
>
> When resources are low on the namenode, it enters SafeMode with a message like this:
> {code}
> 14:00:10,666 WARN NameNodeResourceChecker:89 - Space available on
> volume '/dev/hda1' is 104357888, which is below the configured
> reserved amount 104857600
> 14:00:10,670 WARN FSNamesystem:3190 - NameNode low on available disk
> space. Entering safe mode.
> 14:00:10,670 INFO StateChange:3836 - STATE* Safe mode is ON.
> Resources are low on NN. Safe mode must be turned off manually.
> {code}
> However, turning off safe mode manually has no effect, since it immediately puts itself into Safe Mode again with a log message like this:
> {code}
> 14:00:10,666 WARN NameNodeResourceChecker:89 - Space available on
> volume '/dev/hda1' is 104357888, which is below the configured
> reserved amount 104857600
> 14:00:10,670 WARN FSNamesystem:3190 - NameNode low on available disk
> space. Entering safe mode.
> 14:00:10,670 INFO StateChange:3836 - STATE* Safe mode is ON.
> Resources are low on NN. Safe mode must be turned off manually.
> {code}
> From the shell, it looks like this:
> {code}
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode get
> Safe mode is ON
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode leave
> Safe mode is OFF
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode get
> Safe mode is ON
> {code}
> It seems like we should change the message about turning off safe mode manually, if turning off safe mode manually does not actually work for the case where resources are low. Probably we need to explain that safe mode should be turned off manually *after* adding more resources. As it is, the error message seems misleading.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4021) Misleading error message when
resources are low on the NameNode
Posted by "Christopher Conner (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Christopher Conner updated HDFS-4021:
-------------------------------------
Attachment: HDFS-4021.patch
Changed warning message from:
Resources are low on NN. Safe mode must be turned off manually.
To:
Resources are low on NN. Please add more resources then turn off safe mode manually. NOTE: If you turn off safe mode before adding resources, the NN will immediately return to safe mode.
> Misleading error message when resources are low on the NameNode
> ---------------------------------------------------------------
>
> Key: HDFS-4021
> URL: https://issues.apache.org/jira/browse/HDFS-4021
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Priority: Minor
> Labels: newbie
> Attachments: HDFS-4021.patch
>
>
> When resources are low on the namenode, it enters SafeMode with a message like this:
> {code}
> 14:00:10,666 WARN NameNodeResourceChecker:89 - Space available on
> volume '/dev/hda1' is 104357888, which is below the configured
> reserved amount 104857600
> 14:00:10,670 WARN FSNamesystem:3190 - NameNode low on available disk
> space. Entering safe mode.
> 14:00:10,670 INFO StateChange:3836 - STATE* Safe mode is ON.
> Resources are low on NN. Safe mode must be turned off manually.
> {code}
> However, turning off safe mode manually has no effect, since it immediately puts itself into Safe Mode again with a log message like this:
> {code}
> 14:00:10,666 WARN NameNodeResourceChecker:89 - Space available on
> volume '/dev/hda1' is 104357888, which is below the configured
> reserved amount 104857600
> 14:00:10,670 WARN FSNamesystem:3190 - NameNode low on available disk
> space. Entering safe mode.
> 14:00:10,670 INFO StateChange:3836 - STATE* Safe mode is ON.
> Resources are low on NN. Safe mode must be turned off manually.
> {code}
> From the shell, it looks like this:
> {code}
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode get
> Safe mode is ON
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode leave
> Safe mode is OFF
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode get
> Safe mode is ON
> {code}
> It seems like we should change the message about turning off safe mode manually, if turning off safe mode manually does not actually work for the case where resources are low. Probably we need to explain that safe mode should be turned off manually *after* adding more resources. As it is, the error message seems misleading.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4021) Misleading error message when
resources are low on the NameNode
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13473812#comment-13473812 ]
Hudson commented on HDFS-4021:
------------------------------
Integrated in Hadoop-Common-trunk-Commit #2843 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2843/])
HDFS-4021. Misleading error message when resources are low on the NameNode. Contributed by Christopher Conner. (Revision 1396881)
Result = SUCCESS
atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1396881
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
> Misleading error message when resources are low on the NameNode
> ---------------------------------------------------------------
>
> Key: HDFS-4021
> URL: https://issues.apache.org/jira/browse/HDFS-4021
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Christopher Conner
> Priority: Minor
> Labels: newbie
> Fix For: 2.0.3-alpha
>
> Attachments: HDFS-4021.patch, HDFS-4021.patch
>
>
> When resources are low on the namenode, it enters SafeMode with a message like this:
> {code}
> 14:00:10,666 WARN NameNodeResourceChecker:89 - Space available on
> volume '/dev/hda1' is 104357888, which is below the configured
> reserved amount 104857600
> 14:00:10,670 WARN FSNamesystem:3190 - NameNode low on available disk
> space. Entering safe mode.
> 14:00:10,670 INFO StateChange:3836 - STATE* Safe mode is ON.
> Resources are low on NN. Safe mode must be turned off manually.
> {code}
> However, turning off safe mode manually has no effect, since it immediately puts itself into Safe Mode again with a log message like this:
> {code}
> 14:00:10,666 WARN NameNodeResourceChecker:89 - Space available on
> volume '/dev/hda1' is 104357888, which is below the configured
> reserved amount 104857600
> 14:00:10,670 WARN FSNamesystem:3190 - NameNode low on available disk
> space. Entering safe mode.
> 14:00:10,670 INFO StateChange:3836 - STATE* Safe mode is ON.
> Resources are low on NN. Safe mode must be turned off manually.
> {code}
> From the shell, it looks like this:
> {code}
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode get
> Safe mode is ON
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode leave
> Safe mode is OFF
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode get
> Safe mode is ON
> {code}
> It seems like we should change the message about turning off safe mode manually, if turning off safe mode manually does not actually work for the case where resources are low. Probably we need to explain that safe mode should be turned off manually *after* adding more resources. As it is, the error message seems misleading.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4021) Misleading error message when
resources are low on the NameNode
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472873#comment-13472873 ]
Hadoop QA commented on HDFS-4021:
---------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12548477/HDFS-4021.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/3296//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3296//console
This message is automatically generated.
> Misleading error message when resources are low on the NameNode
> ---------------------------------------------------------------
>
> Key: HDFS-4021
> URL: https://issues.apache.org/jira/browse/HDFS-4021
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Priority: Minor
> Labels: newbie
> Attachments: HDFS-4021.patch
>
>
> When resources are low on the namenode, it enters SafeMode with a message like this:
> {code}
> 14:00:10,666 WARN NameNodeResourceChecker:89 - Space available on
> volume '/dev/hda1' is 104357888, which is below the configured
> reserved amount 104857600
> 14:00:10,670 WARN FSNamesystem:3190 - NameNode low on available disk
> space. Entering safe mode.
> 14:00:10,670 INFO StateChange:3836 - STATE* Safe mode is ON.
> Resources are low on NN. Safe mode must be turned off manually.
> {code}
> However, turning off safe mode manually has no effect, since it immediately puts itself into Safe Mode again with a log message like this:
> {code}
> 14:00:10,666 WARN NameNodeResourceChecker:89 - Space available on
> volume '/dev/hda1' is 104357888, which is below the configured
> reserved amount 104857600
> 14:00:10,670 WARN FSNamesystem:3190 - NameNode low on available disk
> space. Entering safe mode.
> 14:00:10,670 INFO StateChange:3836 - STATE* Safe mode is ON.
> Resources are low on NN. Safe mode must be turned off manually.
> {code}
> From the shell, it looks like this:
> {code}
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode get
> Safe mode is ON
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode leave
> Safe mode is OFF
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode get
> Safe mode is ON
> {code}
> It seems like we should change the message about turning off safe mode manually, if turning off safe mode manually does not actually work for the case where resources are low. Probably we need to explain that safe mode should be turned off manually *after* adding more resources. As it is, the error message seems misleading.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HDFS-4021) Misleading error message when
resources are low on the NameNode
Posted by "Colin Patrick McCabe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Colin Patrick McCabe reassigned HDFS-4021:
------------------------------------------
Assignee: Colin Patrick McCabe
> Misleading error message when resources are low on the NameNode
> ---------------------------------------------------------------
>
> Key: HDFS-4021
> URL: https://issues.apache.org/jira/browse/HDFS-4021
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Priority: Minor
> Labels: newbie
> Attachments: HDFS-4021.patch
>
>
> When resources are low on the namenode, it enters SafeMode with a message like this:
> {code}
> 14:00:10,666 WARN NameNodeResourceChecker:89 - Space available on
> volume '/dev/hda1' is 104357888, which is below the configured
> reserved amount 104857600
> 14:00:10,670 WARN FSNamesystem:3190 - NameNode low on available disk
> space. Entering safe mode.
> 14:00:10,670 INFO StateChange:3836 - STATE* Safe mode is ON.
> Resources are low on NN. Safe mode must be turned off manually.
> {code}
> However, turning off safe mode manually has no effect, since it immediately puts itself into Safe Mode again with a log message like this:
> {code}
> 14:00:10,666 WARN NameNodeResourceChecker:89 - Space available on
> volume '/dev/hda1' is 104357888, which is below the configured
> reserved amount 104857600
> 14:00:10,670 WARN FSNamesystem:3190 - NameNode low on available disk
> space. Entering safe mode.
> 14:00:10,670 INFO StateChange:3836 - STATE* Safe mode is ON.
> Resources are low on NN. Safe mode must be turned off manually.
> {code}
> From the shell, it looks like this:
> {code}
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode get
> Safe mode is ON
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode leave
> Safe mode is OFF
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode get
> Safe mode is ON
> {code}
> It seems like we should change the message about turning off safe mode manually, if turning off safe mode manually does not actually work for the case where resources are low. Probably we need to explain that safe mode should be turned off manually *after* adding more resources. As it is, the error message seems misleading.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4021) Misleading error message when
resources are low on the NameNode
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13474123#comment-13474123 ]
Hudson commented on HDFS-4021:
------------------------------
Integrated in Hadoop-Hdfs-trunk #1192 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1192/])
HDFS-4021. Misleading error message when resources are low on the NameNode. Contributed by Christopher Conner. (Revision 1396881)
Result = SUCCESS
atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1396881
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
> Misleading error message when resources are low on the NameNode
> ---------------------------------------------------------------
>
> Key: HDFS-4021
> URL: https://issues.apache.org/jira/browse/HDFS-4021
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Christopher Conner
> Priority: Minor
> Labels: newbie
> Fix For: 2.0.3-alpha
>
> Attachments: HDFS-4021.patch, HDFS-4021.patch
>
>
> When resources are low on the namenode, it enters SafeMode with a message like this:
> {code}
> 14:00:10,666 WARN NameNodeResourceChecker:89 - Space available on
> volume '/dev/hda1' is 104357888, which is below the configured
> reserved amount 104857600
> 14:00:10,670 WARN FSNamesystem:3190 - NameNode low on available disk
> space. Entering safe mode.
> 14:00:10,670 INFO StateChange:3836 - STATE* Safe mode is ON.
> Resources are low on NN. Safe mode must be turned off manually.
> {code}
> However, turning off safe mode manually has no effect, since it immediately puts itself into Safe Mode again with a log message like this:
> {code}
> 14:00:10,666 WARN NameNodeResourceChecker:89 - Space available on
> volume '/dev/hda1' is 104357888, which is below the configured
> reserved amount 104857600
> 14:00:10,670 WARN FSNamesystem:3190 - NameNode low on available disk
> space. Entering safe mode.
> 14:00:10,670 INFO StateChange:3836 - STATE* Safe mode is ON.
> Resources are low on NN. Safe mode must be turned off manually.
> {code}
> From the shell, it looks like this:
> {code}
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode get
> Safe mode is ON
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode leave
> Safe mode is OFF
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode get
> Safe mode is ON
> {code}
> It seems like we should change the message about turning off safe mode manually, if turning off safe mode manually does not actually work for the case where resources are low. Probably we need to explain that safe mode should be turned off manually *after* adding more resources. As it is, the error message seems misleading.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4021) Misleading error message when
resources are low on the NameNode
Posted by "Christopher Conner (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13473745#comment-13473745 ]
Christopher Conner commented on HDFS-4021:
------------------------------------------
Thanks for getting it committed!!
> Misleading error message when resources are low on the NameNode
> ---------------------------------------------------------------
>
> Key: HDFS-4021
> URL: https://issues.apache.org/jira/browse/HDFS-4021
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Christopher Conner
> Priority: Minor
> Labels: newbie
> Fix For: 2.0.3-alpha
>
> Attachments: HDFS-4021.patch, HDFS-4021.patch
>
>
> When resources are low on the namenode, it enters SafeMode with a message like this:
> {code}
> 14:00:10,666 WARN NameNodeResourceChecker:89 - Space available on
> volume '/dev/hda1' is 104357888, which is below the configured
> reserved amount 104857600
> 14:00:10,670 WARN FSNamesystem:3190 - NameNode low on available disk
> space. Entering safe mode.
> 14:00:10,670 INFO StateChange:3836 - STATE* Safe mode is ON.
> Resources are low on NN. Safe mode must be turned off manually.
> {code}
> However, turning off safe mode manually has no effect, since it immediately puts itself into Safe Mode again with a log message like this:
> {code}
> 14:00:10,666 WARN NameNodeResourceChecker:89 - Space available on
> volume '/dev/hda1' is 104357888, which is below the configured
> reserved amount 104857600
> 14:00:10,670 WARN FSNamesystem:3190 - NameNode low on available disk
> space. Entering safe mode.
> 14:00:10,670 INFO StateChange:3836 - STATE* Safe mode is ON.
> Resources are low on NN. Safe mode must be turned off manually.
> {code}
> From the shell, it looks like this:
> {code}
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode get
> Safe mode is ON
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode leave
> Safe mode is OFF
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode get
> Safe mode is ON
> {code}
> It seems like we should change the message about turning off safe mode manually, if turning off safe mode manually does not actually work for the case where resources are low. Probably we need to explain that safe mode should be turned off manually *after* adding more resources. As it is, the error message seems misleading.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4021) Misleading error message when
resources are low on the NameNode
Posted by "Christopher Conner (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13473541#comment-13473541 ]
Christopher Conner commented on HDFS-4021:
------------------------------------------
Yup, I think that makes more sense. I'll go ahead and make that change.
Thanks!
> Misleading error message when resources are low on the NameNode
> ---------------------------------------------------------------
>
> Key: HDFS-4021
> URL: https://issues.apache.org/jira/browse/HDFS-4021
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Christopher Conner
> Priority: Minor
> Labels: newbie
> Attachments: HDFS-4021.patch
>
>
> When resources are low on the namenode, it enters SafeMode with a message like this:
> {code}
> 14:00:10,666 WARN NameNodeResourceChecker:89 - Space available on
> volume '/dev/hda1' is 104357888, which is below the configured
> reserved amount 104857600
> 14:00:10,670 WARN FSNamesystem:3190 - NameNode low on available disk
> space. Entering safe mode.
> 14:00:10,670 INFO StateChange:3836 - STATE* Safe mode is ON.
> Resources are low on NN. Safe mode must be turned off manually.
> {code}
> However, turning off safe mode manually has no effect, since it immediately puts itself into Safe Mode again with a log message like this:
> {code}
> 14:00:10,666 WARN NameNodeResourceChecker:89 - Space available on
> volume '/dev/hda1' is 104357888, which is below the configured
> reserved amount 104857600
> 14:00:10,670 WARN FSNamesystem:3190 - NameNode low on available disk
> space. Entering safe mode.
> 14:00:10,670 INFO StateChange:3836 - STATE* Safe mode is ON.
> Resources are low on NN. Safe mode must be turned off manually.
> {code}
> From the shell, it looks like this:
> {code}
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode get
> Safe mode is ON
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode leave
> Safe mode is OFF
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode get
> Safe mode is ON
> {code}
> It seems like we should change the message about turning off safe mode manually, if turning off safe mode manually does not actually work for the case where resources are low. Probably we need to explain that safe mode should be turned off manually *after* adding more resources. As it is, the error message seems misleading.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4021) Misleading error message when
resources are low on the NameNode
Posted by "Colin Patrick McCabe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13473485#comment-13473485 ]
Colin Patrick McCabe commented on HDFS-4021:
--------------------------------------------
With regard Jenkins output: to the I don't think tests are needed for this, since it purely changes a log message.
Looks good to me.
> Misleading error message when resources are low on the NameNode
> ---------------------------------------------------------------
>
> Key: HDFS-4021
> URL: https://issues.apache.org/jira/browse/HDFS-4021
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Christopher Conner
> Priority: Minor
> Labels: newbie
> Attachments: HDFS-4021.patch
>
>
> When resources are low on the namenode, it enters SafeMode with a message like this:
> {code}
> 14:00:10,666 WARN NameNodeResourceChecker:89 - Space available on
> volume '/dev/hda1' is 104357888, which is below the configured
> reserved amount 104857600
> 14:00:10,670 WARN FSNamesystem:3190 - NameNode low on available disk
> space. Entering safe mode.
> 14:00:10,670 INFO StateChange:3836 - STATE* Safe mode is ON.
> Resources are low on NN. Safe mode must be turned off manually.
> {code}
> However, turning off safe mode manually has no effect, since it immediately puts itself into Safe Mode again with a log message like this:
> {code}
> 14:00:10,666 WARN NameNodeResourceChecker:89 - Space available on
> volume '/dev/hda1' is 104357888, which is below the configured
> reserved amount 104857600
> 14:00:10,670 WARN FSNamesystem:3190 - NameNode low on available disk
> space. Entering safe mode.
> 14:00:10,670 INFO StateChange:3836 - STATE* Safe mode is ON.
> Resources are low on NN. Safe mode must be turned off manually.
> {code}
> From the shell, it looks like this:
> {code}
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode get
> Safe mode is ON
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode leave
> Safe mode is OFF
> [cmccabe@vm1 h]$ ./bin/hdfs dfsadmin -safemode get
> Safe mode is ON
> {code}
> It seems like we should change the message about turning off safe mode manually, if turning off safe mode manually does not actually work for the case where resources are low. Probably we need to explain that safe mode should be turned off manually *after* adding more resources. As it is, the error message seems misleading.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira