You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "dhruba borthakur (JIRA)" <ji...@apache.org> on 2007/06/15 23:29:26 UTC

[jira] Created: (HADOOP-1497) Possibility of duplicate blockids if dead-datanodes come back up after corresponding files were deleted

Possibility of duplicate blockids if dead-datanodes come back up after corresponding files were deleted
-------------------------------------------------------------------------------------------------------

                 Key: HADOOP-1497
                 URL: https://issues.apache.org/jira/browse/HADOOP-1497
             Project: Hadoop
          Issue Type: Bug
            Reporter: dhruba borthakur
            Assignee: dhruba borthakur


Suppose a datanode D has a block B that belongs to file F. Suppose the datanode D dies and the namenode replicates those blocks to other datanodes. No, suppose the user deletes file F. The namenode removes all the blocks that belonged to file F. Now, suppose a new file F1 is created and the namenode generates the same blockid B for this new file F1. 

Suppose the old datanode D comes back to life. Now we have a valid corrupted block B on datanode D.

This case is possibly detected by the Client (using CRC). But does HDFS need to handle this scenario better?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-1497) Possibility of duplicate blockids if dead-datanodes come back up after corresponding files were deleted

Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-1497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12505419 ] 

Konstantin Shvachko commented on HADOOP-1497:
---------------------------------------------

See HADOOP-158

> Possibility of duplicate blockids if dead-datanodes come back up after corresponding files were deleted
> -------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-1497
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1497
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>            Reporter: dhruba borthakur
>            Assignee: dhruba borthakur
>
> Suppose a datanode D has a block B that belongs to file F. Suppose the datanode D dies and the namenode replicates those blocks to other datanodes. No, suppose the user deletes file F. The namenode removes all the blocks that belonged to file F. Now, suppose a new file F1 is created and the namenode generates the same blockid B for this new file F1. 
> Suppose the old datanode D comes back to life. Now we have a valid corrupted block B on datanode D.
> This case is possibly detected by the Client (using CRC). But does HDFS need to handle this scenario better?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HADOOP-1497) Possibility of duplicate blockids if dead-datanodes come back up after corresponding files were deleted

Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-1497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

dhruba borthakur updated HADOOP-1497:
-------------------------------------

    Component/s: dfs

> Possibility of duplicate blockids if dead-datanodes come back up after corresponding files were deleted
> -------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-1497
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1497
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>            Reporter: dhruba borthakur
>            Assignee: dhruba borthakur
>
> Suppose a datanode D has a block B that belongs to file F. Suppose the datanode D dies and the namenode replicates those blocks to other datanodes. No, suppose the user deletes file F. The namenode removes all the blocks that belonged to file F. Now, suppose a new file F1 is created and the namenode generates the same blockid B for this new file F1. 
> Suppose the old datanode D comes back to life. Now we have a valid corrupted block B on datanode D.
> This case is possibly detected by the Client (using CRC). But does HDFS need to handle this scenario better?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HADOOP-1497) Possibility of duplicate blockids if dead-datanodes come back up after corresponding files were deleted

Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-1497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

dhruba borthakur updated HADOOP-1497:
-------------------------------------

    Fix Version/s: 0.15.0

> Possibility of duplicate blockids if dead-datanodes come back up after corresponding files were deleted
> -------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-1497
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1497
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>            Reporter: dhruba borthakur
>            Assignee: dhruba borthakur
>             Fix For: 0.15.0
>
>
> Suppose a datanode D has a block B that belongs to file F. Suppose the datanode D dies and the namenode replicates those blocks to other datanodes. No, suppose the user deletes file F. The namenode removes all the blocks that belonged to file F. Now, suppose a new file F1 is created and the namenode generates the same blockid B for this new file F1. 
> Suppose the old datanode D comes back to life. Now we have a valid corrupted block B on datanode D.
> This case is possibly detected by the Client (using CRC). But does HDFS need to handle this scenario better?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-1497) Possibility of duplicate blockids if dead-datanodes come back up after corresponding files were deleted

Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-1497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12505420 ] 

Raghu Angadi commented on HADOOP-1497:
--------------------------------------

> This case is possibly detected by the Client (using CRC).
Actually with Block-Level CRCs this may not be detected at the client either . Since crc is per block, a client will silently get wrong data.


> Possibility of duplicate blockids if dead-datanodes come back up after corresponding files were deleted
> -------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-1497
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1497
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>            Reporter: dhruba borthakur
>            Assignee: dhruba borthakur
>
> Suppose a datanode D has a block B that belongs to file F. Suppose the datanode D dies and the namenode replicates those blocks to other datanodes. No, suppose the user deletes file F. The namenode removes all the blocks that belonged to file F. Now, suppose a new file F1 is created and the namenode generates the same blockid B for this new file F1. 
> Suppose the old datanode D comes back to life. Now we have a valid corrupted block B on datanode D.
> This case is possibly detected by the Client (using CRC). But does HDFS need to handle this scenario better?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.