You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-dev@hadoop.apache.org by "Hairong Kuang (JIRA)" <ji...@apache.org> on 2009/03/23 18:51:51 UTC

[jira] Created: (HADOOP-5554) DataNodeCluster should create blocks with the same generation stamp as the blocks created in CreateEditsLog

DataNodeCluster should create blocks with the same generation stamp as the blocks created in CreateEditsLog
-----------------------------------------------------------------------------------------------------------

                 Key: HADOOP-5554
                 URL: https://issues.apache.org/jira/browse/HADOOP-5554
             Project: Hadoop Core
          Issue Type: Bug
    Affects Versions: 0.19.1
            Reporter: Hairong Kuang
            Assignee: Hairong Kuang
             Fix For: 0.19.2, 0.20.0, 0.21.0


HADOOP-5384 makes DataNodeCluster to create blocks with generation stamp Block#GRANDFATHER_GENERATION_STAMP(0) so simuated datanodes do not crash NameNode any more. But there is still a problem. CreateEditLogs creates blocks with generation stamp GenerationStamp#FIRST_VALID_STAMP (1000). Because of the generation stamp mismatch, all injected blocks are marked as invalid when NameNode processes block reports.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (HADOOP-5554) DataNodeCluster should create blocks with the same generation stamp as the blocks created in CreateEditsLog

Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.

     [ https://issues.apache.org/jira/browse/HADOOP-5554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hairong Kuang updated HADOOP-5554:
----------------------------------

    Attachment: blockGS.patch

This patch makes both CreateEditsLog and DataNodeCluster to create blocks with generation stamp GenerationStamp#FIRST_VALID_STAMP.

> DataNodeCluster should create blocks with the same generation stamp as the blocks created in CreateEditsLog
> -----------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-5554
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5554
>             Project: Hadoop Core
>          Issue Type: Bug
>    Affects Versions: 0.19.1
>            Reporter: Hairong Kuang
>            Assignee: Hairong Kuang
>             Fix For: 0.19.2, 0.20.0, 0.21.0
>
>         Attachments: blockGS.patch
>
>
> HADOOP-5384 makes DataNodeCluster to create blocks with generation stamp Block#GRANDFATHER_GENERATION_STAMP(0) so simuated datanodes do not crash NameNode any more. But there is still a problem. CreateEditLogs creates blocks with generation stamp GenerationStamp#FIRST_VALID_STAMP (1000). Because of the generation stamp mismatch, all injected blocks are marked as invalid when NameNode processes block reports.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (HADOOP-5554) DataNodeCluster should create blocks with the same generation stamp as the blocks created in CreateEditsLog

Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.

     [ https://issues.apache.org/jira/browse/HADOOP-5554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hairong Kuang updated HADOOP-5554:
----------------------------------

    Affects Version/s:     (was: 0.19.1)
                       0.19.2
               Status: Patch Available  (was: Open)

> DataNodeCluster should create blocks with the same generation stamp as the blocks created in CreateEditsLog
> -----------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-5554
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5554
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: test
>    Affects Versions: 0.19.2
>            Reporter: Hairong Kuang
>            Assignee: Hairong Kuang
>             Fix For: 0.19.2, 0.20.0, 0.21.0
>
>         Attachments: blockGS.patch
>
>
> HADOOP-5384 makes DataNodeCluster to create blocks with generation stamp Block#GRANDFATHER_GENERATION_STAMP(0) so simuated datanodes do not crash NameNode any more. But there is still a problem. CreateEditLogs creates blocks with generation stamp GenerationStamp#FIRST_VALID_STAMP (1000). Because of the generation stamp mismatch, all injected blocks are marked as invalid when NameNode processes block reports.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (HADOOP-5554) DataNodeCluster should create blocks with the same generation stamp as the blocks created in CreateEditsLog

Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.

     [ https://issues.apache.org/jira/browse/HADOOP-5554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hairong Kuang updated HADOOP-5554:
----------------------------------

    Attachment: blockGS-0.19.patch

This is a patch for 0.19.

> DataNodeCluster should create blocks with the same generation stamp as the blocks created in CreateEditsLog
> -----------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-5554
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5554
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: test
>    Affects Versions: 0.19.2
>            Reporter: Hairong Kuang
>            Assignee: Hairong Kuang
>             Fix For: 0.19.2, 0.20.0, 0.21.0
>
>         Attachments: blockGS-0.19.patch, blockGS.patch
>
>
> HADOOP-5384 makes DataNodeCluster to create blocks with generation stamp Block#GRANDFATHER_GENERATION_STAMP(0) so simuated datanodes do not crash NameNode any more. But there is still a problem. CreateEditLogs creates blocks with generation stamp GenerationStamp#FIRST_VALID_STAMP (1000). Because of the generation stamp mismatch, all injected blocks are marked as invalid when NameNode processes block reports.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (HADOOP-5554) DataNodeCluster should create blocks with the same generation stamp as the blocks created in CreateEditsLog

Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.

     [ https://issues.apache.org/jira/browse/HADOOP-5554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tsz Wo (Nicholas), SZE updated HADOOP-5554:
-------------------------------------------

     Component/s: test
    Hadoop Flags: [Reviewed]

+1

> DataNodeCluster should create blocks with the same generation stamp as the blocks created in CreateEditsLog
> -----------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-5554
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5554
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: test
>    Affects Versions: 0.19.1
>            Reporter: Hairong Kuang
>            Assignee: Hairong Kuang
>             Fix For: 0.19.2, 0.20.0, 0.21.0
>
>         Attachments: blockGS.patch
>
>
> HADOOP-5384 makes DataNodeCluster to create blocks with generation stamp Block#GRANDFATHER_GENERATION_STAMP(0) so simuated datanodes do not crash NameNode any more. But there is still a problem. CreateEditLogs creates blocks with generation stamp GenerationStamp#FIRST_VALID_STAMP (1000). Because of the generation stamp mismatch, all injected blocks are marked as invalid when NameNode processes block reports.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HADOOP-5554) DataNodeCluster should create blocks with the same generation stamp as the blocks created in CreateEditsLog

Posted by "Hudson (JIRA)" <ji...@apache.org>.

    [ https://issues.apache.org/jira/browse/HADOOP-5554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12688819#action_12688819 ] 

Hudson commented on HADOOP-5554:
--------------------------------

Integrated in Hadoop-trunk #789 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/789/])
    . DataNodeCluster and CreateEditsLog should create blocks with the same generation stamp value. (hairong via szetszwo)


> DataNodeCluster should create blocks with the same generation stamp as the blocks created in CreateEditsLog
> -----------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-5554
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5554
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: test
>    Affects Versions: 0.19.2
>            Reporter: Hairong Kuang
>            Assignee: Hairong Kuang
>             Fix For: 0.19.2, 0.20.0, 0.21.0
>
>         Attachments: blockGS-0.19.patch, blockGS.patch
>
>
> HADOOP-5384 makes DataNodeCluster to create blocks with generation stamp Block#GRANDFATHER_GENERATION_STAMP(0) so simuated datanodes do not crash NameNode any more. But there is still a problem. CreateEditLogs creates blocks with generation stamp GenerationStamp#FIRST_VALID_STAMP (1000). Because of the generation stamp mismatch, all injected blocks are marked as invalid when NameNode processes block reports.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HADOOP-5554) DataNodeCluster should create blocks with the same generation stamp as the blocks created in CreateEditsLog

Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.

    [ https://issues.apache.org/jira/browse/HADOOP-5554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12688458#action_12688458 ] 

Hairong Kuang commented on HADOOP-5554:
---------------------------------------

I have tested the patch manually.

> DataNodeCluster should create blocks with the same generation stamp as the blocks created in CreateEditsLog
> -----------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-5554
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5554
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: test
>    Affects Versions: 0.19.2
>            Reporter: Hairong Kuang
>            Assignee: Hairong Kuang
>             Fix For: 0.19.2, 0.20.0, 0.21.0
>
>         Attachments: blockGS.patch
>
>
> HADOOP-5384 makes DataNodeCluster to create blocks with generation stamp Block#GRANDFATHER_GENERATION_STAMP(0) so simuated datanodes do not crash NameNode any more. But there is still a problem. CreateEditLogs creates blocks with generation stamp GenerationStamp#FIRST_VALID_STAMP (1000). Because of the generation stamp mismatch, all injected blocks are marked as invalid when NameNode processes block reports.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (HADOOP-5554) DataNodeCluster should create blocks with the same generation stamp as the blocks created in CreateEditsLog

Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.

     [ https://issues.apache.org/jira/browse/HADOOP-5554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tsz Wo (Nicholas), SZE updated HADOOP-5554:
-------------------------------------------

    Resolution: Fixed
        Status: Resolved  (was: Patch Available)

I have committed this to 0.19 and above.  Thanks, Hairong!

> DataNodeCluster should create blocks with the same generation stamp as the blocks created in CreateEditsLog
> -----------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-5554
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5554
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: test
>    Affects Versions: 0.19.2
>            Reporter: Hairong Kuang
>            Assignee: Hairong Kuang
>             Fix For: 0.19.2, 0.20.0, 0.21.0
>
>         Attachments: blockGS-0.19.patch, blockGS.patch
>
>
> HADOOP-5384 makes DataNodeCluster to create blocks with generation stamp Block#GRANDFATHER_GENERATION_STAMP(0) so simuated datanodes do not crash NameNode any more. But there is still a problem. CreateEditLogs creates blocks with generation stamp GenerationStamp#FIRST_VALID_STAMP (1000). Because of the generation stamp mismatch, all injected blocks are marked as invalid when NameNode processes block reports.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HADOOP-5554) DataNodeCluster should create blocks with the same generation stamp as the blocks created in CreateEditsLog

Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.

    [ https://issues.apache.org/jira/browse/HADOOP-5554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12688404#action_12688404 ] 

Tsz Wo (Nicholas), SZE commented on HADOOP-5554:
------------------------------------------------

This patch changes tools for manual tests.  I think we don't have junit for these test tools.  So, I suggest to do some manual tests instead of waiting for Hudson.

> DataNodeCluster should create blocks with the same generation stamp as the blocks created in CreateEditsLog
> -----------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-5554
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5554
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: test
>    Affects Versions: 0.19.2
>            Reporter: Hairong Kuang
>            Assignee: Hairong Kuang
>             Fix For: 0.19.2, 0.20.0, 0.21.0
>
>         Attachments: blockGS.patch
>
>
> HADOOP-5384 makes DataNodeCluster to create blocks with generation stamp Block#GRANDFATHER_GENERATION_STAMP(0) so simuated datanodes do not crash NameNode any more. But there is still a problem. CreateEditLogs creates blocks with generation stamp GenerationStamp#FIRST_VALID_STAMP (1000). Because of the generation stamp mismatch, all injected blocks are marked as invalid when NameNode processes block reports.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.