You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org> on 2008/06/17 02:15:44 UTC

[jira] Created: (HADOOP-3574) Better Datanode DiskOutOfSpaceException handling.

Better Datanode DiskOutOfSpaceException handling.
-------------------------------------------------

                 Key: HADOOP-3574
                 URL: https://issues.apache.org/jira/browse/HADOOP-3574
             Project: Hadoop Core
          Issue Type: Improvement
          Components: dfs
            Reporter: Tsz Wo (Nicholas), SZE
             Fix For: 0.19.0


When a datanode ran out of disk space, it throws DiskOutOfSpaceException internally.  However, it does not notify the writing client about the exception.  As a result, the client will wait until socket timeout.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (HADOOP-3574) Better Datanode DiskOutOfSpaceException handling.

Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-3574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Devaraj Das updated HADOOP-3574:
--------------------------------

    Fix Version/s:     (was: 0.19.0)

> Better Datanode DiskOutOfSpaceException handling.
> -------------------------------------------------
>
>                 Key: HADOOP-3574
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3574
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: dfs
>            Reporter: Tsz Wo (Nicholas), SZE
>
> When a datanode ran out of disk space, it throws DiskOutOfSpaceException internally.  However, it does not notify the writing client about the exception.  As a result, the client will wait until socket timeout.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HADOOP-3574) Better Datanode DiskOutOfSpaceException handling.

Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-3574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12650754#action_12650754 ] 

Raghu Angadi commented on HADOOP-3574:
--------------------------------------

Informing others won't hurt . but HDFS should not depend on that for any important functionality since this Datanode is already in bad state.. we can not expect it to do much.



> Better Datanode DiskOutOfSpaceException handling.
> -------------------------------------------------
>
>                 Key: HADOOP-3574
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3574
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: dfs
>            Reporter: Tsz Wo (Nicholas), SZE
>
> When a datanode ran out of disk space, it throws DiskOutOfSpaceException internally.  However, it does not notify the writing client about the exception.  As a result, the client will wait until socket timeout.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HADOOP-3574) Better Datanode DiskOutOfSpaceException handling.

Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-3574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12605462#action_12605462 ] 

Tsz Wo (Nicholas), SZE commented on HADOOP-3574:
------------------------------------------------

This problem was observed in HADOOP-3504.

> Better Datanode DiskOutOfSpaceException handling.
> -------------------------------------------------
>
>                 Key: HADOOP-3574
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3574
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: dfs
>            Reporter: Tsz Wo (Nicholas), SZE
>             Fix For: 0.19.0
>
>
> When a datanode ran out of disk space, it throws DiskOutOfSpaceException internally.  However, it does not notify the writing client about the exception.  As a result, the client will wait until socket timeout.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HADOOP-3574) Better Datanode DiskOutOfSpaceException handling.

Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-3574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12650681#action_12650681 ] 

Tsz Wo (Nicholas), SZE commented on HADOOP-3574:
------------------------------------------------

Should the out-of-space datanode send an error code to other parties?

> Better Datanode DiskOutOfSpaceException handling.
> -------------------------------------------------
>
>                 Key: HADOOP-3574
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3574
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: dfs
>            Reporter: Tsz Wo (Nicholas), SZE
>
> When a datanode ran out of disk space, it throws DiskOutOfSpaceException internally.  However, it does not notify the writing client about the exception.  As a result, the client will wait until socket timeout.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HADOOP-3574) Better Datanode DiskOutOfSpaceException handling.

Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-3574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12652626#action_12652626 ] 

Tsz Wo (Nicholas), SZE commented on HADOOP-3574:
------------------------------------------------

> ... since this Datanode is already in bad state.. we can not expect it to do much. 

Theoretically, the out-of-space datanode should be able to send an error code to other parties since the bad state is about out of disk space, not network problems.  However, implementing this might be hard.

> Better Datanode DiskOutOfSpaceException handling.
> -------------------------------------------------
>
>                 Key: HADOOP-3574
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3574
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: dfs
>            Reporter: Tsz Wo (Nicholas), SZE
>
> When a datanode ran out of disk space, it throws DiskOutOfSpaceException internally.  However, it does not notify the writing client about the exception.  As a result, the client will wait until socket timeout.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HADOOP-3574) Better Datanode DiskOutOfSpaceException handling.

Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-3574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12650665#action_12650665 ] 

Raghu Angadi commented on HADOOP-3574:
--------------------------------------

Datanode should close the socket in case of any exception including this one.


> Better Datanode DiskOutOfSpaceException handling.
> -------------------------------------------------
>
>                 Key: HADOOP-3574
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3574
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: dfs
>            Reporter: Tsz Wo (Nicholas), SZE
>
> When a datanode ran out of disk space, it throws DiskOutOfSpaceException internally.  However, it does not notify the writing client about the exception.  As a result, the client will wait until socket timeout.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HADOOP-3574) Better Datanode DiskOutOfSpaceException handling.

Posted by "Suresh Srinivas (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-3574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12652684#action_12652684 ] 

Suresh Srinivas commented on HADOOP-3574:
-----------------------------------------

Should we create another jira to recover form this condition. We could use mechanism similar to balancer to move some blocks to other datanodes to free up space? Datanodes could also indicate this condition to namenode and trigger freeing up space by moving blocks. Alternatively, this could also be done proactively by the namenode when the free space reported by the datanode (during heartbeat) goes below certain threshold. This will improve robustness.

> Better Datanode DiskOutOfSpaceException handling.
> -------------------------------------------------
>
>                 Key: HADOOP-3574
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3574
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: dfs
>            Reporter: Tsz Wo (Nicholas), SZE
>
> When a datanode ran out of disk space, it throws DiskOutOfSpaceException internally.  However, it does not notify the writing client about the exception.  As a result, the client will wait until socket timeout.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.