You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "philo vivero (JIRA)" <ji...@apache.org> on 2011/05/19 03:20:47 UTC
[jira] [Created] (HDFS-1960) dfs.*.dir should not default to /tmp
(or other typically volatile storage)
dfs.*.dir should not default to /tmp (or other typically volatile storage)
--------------------------------------------------------------------------
Key: HDFS-1960
URL: https://issues.apache.org/jira/browse/HDFS-1960
Project: Hadoop HDFS
Issue Type: Improvement
Components: data-node
Affects Versions: 0.20.2
Environment: *nix systems
Reporter: philo vivero
Priority: Critical
The hdfs-site.xml file possibly will not define one or both of:
dfs.name.dir
dfs.data.dir
If they are not specified, data is stored in /tmp. This is extremely dangerous. Rationale: the cluster will work fine for days, possibly even weeks, before blocks will start to go missing. Rebooting a datanode on common Linux systems will clear all the data from that node. There is no documented way (that I'm aware of) to recover the situation. The cluster must be completely obliterated and rebuilt from scratch.
Better reactions to the missing configuration parameters:
1. DataNode dies on startup and asks that these parameters be defined.
2. Default is /var/db/hadoop (or some other non-volatile storage location). Naturally, inability to write into that directory leads to DataNode to die on startup, logging error.
The first solution would be most likely preferred by typical enterprise sysadmins. The second solution is suboptimal (since /var/db/hadoop might not be the optimal location for the data) but is still preferable to the current implementation, since it will less often lead to an irretrievably corrupt cluster.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1960) dfs.*.dir should not default to /tmp
(or other typically volatile storage)
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13035910#comment-13035910 ]
Todd Lipcon commented on HDFS-1960:
-----------------------------------
This will likely require some discussion. But, I agree.
Another piece of rationale is that HDFS will not start up without defining fs.defaultFS (nee fs.default.name). So, we already require the user to make some configuration changes before running HDFS - we may as well require them to configure the persistent storage locations.
> dfs.*.dir should not default to /tmp (or other typically volatile storage)
> --------------------------------------------------------------------------
>
> Key: HDFS-1960
> URL: https://issues.apache.org/jira/browse/HDFS-1960
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: data-node
> Affects Versions: 0.20.2
> Environment: *nix systems
> Reporter: philo vivero
> Priority: Critical
>
> The hdfs-site.xml file possibly will not define one or both of:
> dfs.name.dir
> dfs.data.dir
> If they are not specified, data is stored in /tmp. This is extremely dangerous. Rationale: the cluster will work fine for days, possibly even weeks, before blocks will start to go missing. Rebooting a datanode on common Linux systems will clear all the data from that node. There is no documented way (that I'm aware of) to recover the situation. The cluster must be completely obliterated and rebuilt from scratch.
> Better reactions to the missing configuration parameters:
> 1. DataNode dies on startup and asks that these parameters be defined.
> 2. Default is /var/db/hadoop (or some other non-volatile storage location). Naturally, inability to write into that directory leads to DataNode to die on startup, logging error.
> The first solution would be most likely preferred by typical enterprise sysadmins. The second solution is suboptimal (since /var/db/hadoop might not be the optimal location for the data) but is still preferable to the current implementation, since it will less often lead to an irretrievably corrupt cluster.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1960) dfs.*.dir should not default to /tmp
(or other typically volatile storage)
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13035922#comment-13035922 ]
Todd Lipcon commented on HDFS-1960:
-----------------------------------
yes, dfs.name.dir and dfs.data.dir are both equally crucial to persist as part of a deployment.
> dfs.*.dir should not default to /tmp (or other typically volatile storage)
> --------------------------------------------------------------------------
>
> Key: HDFS-1960
> URL: https://issues.apache.org/jira/browse/HDFS-1960
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: data-node
> Affects Versions: 0.20.2
> Environment: *nix systems
> Reporter: philo vivero
> Priority: Critical
>
> The hdfs-site.xml file possibly will not define one or both of:
> dfs.name.dir
> dfs.data.dir
> If they are not specified, data is stored in /tmp. This is extremely dangerous. Rationale: the cluster will work fine for days, possibly even weeks, before blocks will start to go missing. Rebooting a datanode on common Linux systems will clear all the data from that node. There is no documented way (that I'm aware of) to recover the situation. The cluster must be completely obliterated and rebuilt from scratch.
> Better reactions to the missing configuration parameters:
> 1. DataNode dies on startup and asks that these parameters be defined.
> 2. Default is /var/db/hadoop (or some other non-volatile storage location). Naturally, inability to write into that directory leads to DataNode to die on startup, logging error.
> The first solution would be most likely preferred by typical enterprise sysadmins. The second solution is suboptimal (since /var/db/hadoop might not be the optimal location for the data) but is still preferable to the current implementation, since it will less often lead to an irretrievably corrupt cluster.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1960) dfs.*.dir should not default to /tmp
(or other typically volatile storage)
Posted by "philo vivero (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13035916#comment-13035916 ]
philo vivero commented on HDFS-1960:
------------------------------------
I'm not 100% sure, but I believe dfs.name.dir data disappearing will cause the NameNode to become entirely useless after a service restart. At least that is what appeared to happen when I tried it.
> dfs.*.dir should not default to /tmp (or other typically volatile storage)
> --------------------------------------------------------------------------
>
> Key: HDFS-1960
> URL: https://issues.apache.org/jira/browse/HDFS-1960
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: data-node
> Affects Versions: 0.20.2
> Environment: *nix systems
> Reporter: philo vivero
> Priority: Critical
>
> The hdfs-site.xml file possibly will not define one or both of:
> dfs.name.dir
> dfs.data.dir
> If they are not specified, data is stored in /tmp. This is extremely dangerous. Rationale: the cluster will work fine for days, possibly even weeks, before blocks will start to go missing. Rebooting a datanode on common Linux systems will clear all the data from that node. There is no documented way (that I'm aware of) to recover the situation. The cluster must be completely obliterated and rebuilt from scratch.
> Better reactions to the missing configuration parameters:
> 1. DataNode dies on startup and asks that these parameters be defined.
> 2. Default is /var/db/hadoop (or some other non-volatile storage location). Naturally, inability to write into that directory leads to DataNode to die on startup, logging error.
> The first solution would be most likely preferred by typical enterprise sysadmins. The second solution is suboptimal (since /var/db/hadoop might not be the optimal location for the data) but is still preferable to the current implementation, since it will less often lead to an irretrievably corrupt cluster.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1960) dfs.*.dir should not default to /tmp
(or other typically volatile storage)
Posted by "Harsh J (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13182041#comment-13182041 ]
Harsh J commented on HDFS-1960:
-------------------------------
Another perhaps-working default could probably be a relative path "{{.}}".
I think this is an issue that could largely be solved by packaging and better administrator's documentation.
> dfs.*.dir should not default to /tmp (or other typically volatile storage)
> --------------------------------------------------------------------------
>
> Key: HDFS-1960
> URL: https://issues.apache.org/jira/browse/HDFS-1960
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: data-node
> Affects Versions: 0.20.2
> Environment: *nix systems
> Reporter: philo vivero
> Priority: Critical
>
> The hdfs-site.xml file possibly will not define one or both of:
> dfs.name.dir
> dfs.data.dir
> If they are not specified, data is stored in /tmp. This is extremely dangerous. Rationale: the cluster will work fine for days, possibly even weeks, before blocks will start to go missing. Rebooting a datanode on common Linux systems will clear all the data from that node. There is no documented way (that I'm aware of) to recover the situation. The cluster must be completely obliterated and rebuilt from scratch.
> Better reactions to the missing configuration parameters:
> 1. DataNode dies on startup and asks that these parameters be defined.
> 2. Default is /var/db/hadoop (or some other non-volatile storage location). Naturally, inability to write into that directory leads to DataNode to die on startup, logging error.
> The first solution would be most likely preferred by typical enterprise sysadmins. The second solution is suboptimal (since /var/db/hadoop might not be the optimal location for the data) but is still preferable to the current implementation, since it will less often lead to an irretrievably corrupt cluster.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira