You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@whirr.apache.org by "Steve Loughran (JIRA)" <ji...@apache.org> on 2012/09/17 17:08:08 UTC
[jira] [Created] (WHIRR-656) HDFS directory creation can deadlock
if the cluster being brought up is single node & preformatted
Steve Loughran created WHIRR-656:
------------------------------------
Summary: HDFS directory creation can deadlock if the cluster being brought up is single node & preformatted
Key: WHIRR-656
URL: https://issues.apache.org/jira/browse/WHIRR-656
Project: Whirr
Issue Type: Bug
Components: service/hadoop
Affects Versions: 0.9.0
Reporter: Steve Loughran
Priority: Minor
If you have a single node cluster & there already exists a pre-formatted filesystem then the cluster config process can deadlock in dfs directory creation
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-656) HDFS directory creation can deadlock
if the cluster being brought up is single node & preformatted
Posted by "Steve Loughran (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13461396#comment-13461396 ]
Steve Loughran commented on WHIRR-656:
--------------------------------------
Other mitigation strategy: start the hadoop-datanode service/role before looking to see if the host is also running hadoop-namenode
> HDFS directory creation can deadlock if the cluster being brought up is single node & preformatted
> ---------------------------------------------------------------------------------------------------
>
> Key: WHIRR-656
> URL: https://issues.apache.org/jira/browse/WHIRR-656
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.9.0
> Reporter: Steve Loughran
> Priority: Minor
>
> If you have a single node cluster & there already exists a pre-formatted filesystem then the cluster config process can deadlock in dfs directory creation
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-656) HDFS directory creation can deadlock
if the cluster being brought up is single node & preformatted
Posted by "Steve Loughran (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13457685#comment-13457685 ]
Steve Loughran commented on WHIRR-656:
--------------------------------------
This could also happen if the cluster coming up is v. small and there is a DN on the master node, as the block count could still be below the threshold.
> HDFS directory creation can deadlock if the cluster being brought up is single node & preformatted
> ---------------------------------------------------------------------------------------------------
>
> Key: WHIRR-656
> URL: https://issues.apache.org/jira/browse/WHIRR-656
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.9.0
> Reporter: Steve Loughran
> Priority: Minor
>
> If you have a single node cluster & there already exists a pre-formatted filesystem then the cluster config process can deadlock in dfs directory creation
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (WHIRR-656) HDFS directory creation can
deadlock if the cluster being brought up is single node & preformatted
Posted by "Steve Loughran (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13457074#comment-13457074 ]
Steve Loughran edited comment on WHIRR-656 at 9/18/12 2:48 AM:
---------------------------------------------------------------
The install process can deadlock in directory creation
Conditions:
# there already exists a preformatted FS
# the only DN coming up is on the same machine
In this scenario,
# the {{start_namenode()}} function can block as dfsadmin waits to come out of safemode,
# NN will only do that once the block count is high enough
# the DN isn't started until the NN is started
# result: deadlock
Fix: don't create the dir structure if the FS was already there.
was (Author: stevel@apache.org):
The install process can deadlock in directory creation
Conditions:
#there already exists a preformatted FS
#the only DN coming up is on the same machine
In this scenario,
# the {{start_namenode()}} function can block as dfsadmin waits to come out of safemode,
# NN will only do that once the block count is high enough
# the DN isn't started until the NN is started
# result: deadlock
Fix: don't create the dir structure if the FS was already there.
> HDFS directory creation can deadlock if the cluster being brought up is single node & preformatted
> ---------------------------------------------------------------------------------------------------
>
> Key: WHIRR-656
> URL: https://issues.apache.org/jira/browse/WHIRR-656
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.9.0
> Reporter: Steve Loughran
> Priority: Minor
>
> If you have a single node cluster & there already exists a pre-formatted filesystem then the cluster config process can deadlock in dfs directory creation
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-656) HDFS directory creation can deadlock
if the cluster being brought up is single node & preformatted
Posted by "Steve Loughran (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13457074#comment-13457074 ]
Steve Loughran commented on WHIRR-656:
--------------------------------------
The install process can deadlock in directory creation
Conditions:
#there already exists a preformatted FS
#the only DN coming up is on the same machine
In this scenario,
# the {{start_namenode()}} function can block as dfsadmin waits to come out of safemode,
# NN will only do that once the block count is high enough
# the DN isn't started until the NN is started
# result: deadlock
Fix: don't create the dir structure if the FS was already there.
> HDFS directory creation can deadlock if the cluster being brought up is single node & preformatted
> ---------------------------------------------------------------------------------------------------
>
> Key: WHIRR-656
> URL: https://issues.apache.org/jira/browse/WHIRR-656
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.9.0
> Reporter: Steve Loughran
> Priority: Minor
>
> If you have a single node cluster & there already exists a pre-formatted filesystem then the cluster config process can deadlock in dfs directory creation
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira