You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "jiwan (JIRA)" <ji...@apache.org> on 2012/07/26 06:05:34 UTC

[jira] [Created] (HDFS-3729) when datanode is blocked in BlockReceiver.receiveBlock(...) by disk error/pressure, DFSClient is blocked and no timeout mechanism

jiwan created HDFS-3729:
---------------------------

             Summary: when datanode is blocked in BlockReceiver.receiveBlock(...) by disk error/pressure, DFSClient is blocked and no timeout mechanism
                 Key: HDFS-3729
                 URL: https://issues.apache.org/jira/browse/HDFS-3729
             Project: Hadoop HDFS
          Issue Type: Bug
          Components: hdfs client
            Reporter: jiwan


Our hadoop/hbase in taobao.com cluster is blocked by DFSClient somtimes. The reason is disk error or too much load, but HEART_BEAT in PacketResponder is normal, so DFSClient wait forever until method of disk read/write return. I searched issues in jira, and nothing for this issue, we plan to do some work to fix this bug. The innial idea is to add timeout mechanism for the DFSClient write function. Does some guys have comments about this?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HDFS-3729) when datanode is blocked in BlockReceiver.receiveBlock(...) by disk error/pressure, DFSClient is blocked and no timeout mechanism

Posted by "jiwan (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HDFS-3729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

jiwan updated HDFS-3729:
------------------------

    Description: Our hadoop/hbase in taobao.com cluster is blocked by DFSClient somtimes. The reason is disk error or too much load, but HEART_BEAT in PacketResponder is normal, so DFSClient wait forever until method of disk read/write return. I searched issues in jira, and nothing for this issue, we plan to do some work to fix this bug. The initial idea is to add timeout mechanism for the DFSClient write function. Does some guys have comments about this?  (was: Our hadoop/hbase in taobao.com cluster is blocked by DFSClient somtimes. The reason is disk error or too much load, but HEART_BEAT in PacketResponder is normal, so DFSClient wait forever until method of disk read/write return. I searched issues in jira, and nothing for this issue, we plan to do some work to fix this bug. The innial idea is to add timeout mechanism for the DFSClient write function. Does some guys have comments about this?)
    
> when datanode is blocked in BlockReceiver.receiveBlock(...) by disk error/pressure, DFSClient is blocked and no timeout mechanism
> ---------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-3729
>                 URL: https://issues.apache.org/jira/browse/HDFS-3729
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: hdfs client
>            Reporter: jiwan
>
> Our hadoop/hbase in taobao.com cluster is blocked by DFSClient somtimes. The reason is disk error or too much load, but HEART_BEAT in PacketResponder is normal, so DFSClient wait forever until method of disk read/write return. I searched issues in jira, and nothing for this issue, we plan to do some work to fix this bug. The initial idea is to add timeout mechanism for the DFSClient write function. Does some guys have comments about this?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3729) when datanode is blocked in BlockReceiver.receiveBlock(...) by disk error/pressure, DFSClient is blocked and no timeout mechanism

Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-3729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13427006#comment-13427006 ] 

Eli Collins commented on HDFS-3729:
-----------------------------------

What version of Hadoop are you using?
                
> when datanode is blocked in BlockReceiver.receiveBlock(...) by disk error/pressure, DFSClient is blocked and no timeout mechanism
> ---------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-3729
>                 URL: https://issues.apache.org/jira/browse/HDFS-3729
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: hdfs client
>            Reporter: jiwan@taobao.com
>
> Our hadoop/hbase in taobao.com cluster is blocked by DFSClient somtimes. The reason is disk error or too much load, but HEART_BEAT in PacketResponder is normal, so DFSClient wait forever until method of disk read/write return. I searched issues in jira, and nothing for this issue, we plan to do some work to fix this bug. The initial idea is to add timeout mechanism for the DFSClient write function. Does some guys have comments about this?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3729) when datanode is blocked in BlockReceiver.receiveBlock(...) by disk error/pressure, DFSClient is blocked and no timeout mechanism

Posted by "jiwan@taobao.com (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-3729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13427063#comment-13427063 ] 

jiwan@taobao.com commented on HDFS-3729:
----------------------------------------

cdh3u3 and below 
                
> when datanode is blocked in BlockReceiver.receiveBlock(...) by disk error/pressure, DFSClient is blocked and no timeout mechanism
> ---------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-3729
>                 URL: https://issues.apache.org/jira/browse/HDFS-3729
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: hdfs client
>            Reporter: jiwan@taobao.com
>
> Our hadoop/hbase in taobao.com cluster is blocked by DFSClient somtimes. The reason is disk error or too much load, but HEART_BEAT in PacketResponder is normal, so DFSClient wait forever until method of disk read/write return. I searched issues in jira, and nothing for this issue, we plan to do some work to fix this bug. The initial idea is to add timeout mechanism for the DFSClient write function. Does some guys have comments about this?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira