You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "chunhui shen (JIRA)" <ji...@apache.org> on 2012/05/02 14:26:54 UTC

[jira] [Created] (HBASE-5918) Master will block forever when startup if root server died between assign root and assign meta

chunhui shen created HBASE-5918:
-----------------------------------

             Summary: Master will block forever when startup if root server died between assign root and assign meta
                 Key: HBASE-5918
                 URL: https://issues.apache.org/jira/browse/HBASE-5918
             Project: HBase
          Issue Type: Bug
            Reporter: chunhui shen
            Assignee: chunhui shen


When master is initializing, if root server died between assign root and assign meta, master will block at 
HMaster#assignRootAndMeta:{code}assignmentManager.assignMeta();
this.catalogTracker.waitForMeta();{code}
because ServerShutdownHandler is disabled,

So we should enable ServerShutdownHandler after called assignmentManager.assignMeta();

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5918) Master will block forever when startup if root server died between assign root and assign meta

Posted by "stack (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13267243#comment-13267243 ] 

stack commented on HBASE-5918:
------------------------------

@Chunhui I suppose so.... Its just odd have flags set in such different locations... It makes the tracking of stuff difficult.  At a minimum I'd think we'd make a single method that set the flag and then did the call to expireDeadNotExpiredServers so they are grouped... 

What about the call to expireDeadNotExpiredServers that is done twice?  On first call, we'd process possibly the server that was carrying root.  What happens when we call it again later out in finishInitialization?  Could we end up processing same server twice at all?

Thanks. 
                
> Master will block forever when startup if root server died between assign root and assign meta
> ----------------------------------------------------------------------------------------------
>
>                 Key: HBASE-5918
>                 URL: https://issues.apache.org/jira/browse/HBASE-5918
>             Project: HBase
>          Issue Type: Bug
>          Components: master
>    Affects Versions: 0.92.1
>            Reporter: chunhui shen
>            Assignee: chunhui shen
>         Attachments: HBASE-5918.patch, HBASE-5918.patch
>
>
> When master is initializing, if root server died between assign root and assign meta, master will block at 
> HMaster#assignRootAndMeta:{code}assignmentManager.assignMeta();
> this.catalogTracker.waitForMeta();{code}
> because ServerShutdownHandler is disabled,
> So we should enable ServerShutdownHandler after called assignmentManager.assignMeta();

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5918) Master will block forever when startup if root server died between assign root and assign meta

Posted by "ramkrishna.s.vasudevan (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13267175#comment-13267175 ] 

ramkrishna.s.vasudevan commented on HBASE-5918:
-----------------------------------------------

+1 on patch.  
                
> Master will block forever when startup if root server died between assign root and assign meta
> ----------------------------------------------------------------------------------------------
>
>                 Key: HBASE-5918
>                 URL: https://issues.apache.org/jira/browse/HBASE-5918
>             Project: HBase
>          Issue Type: Bug
>          Components: master
>    Affects Versions: 0.92.1
>            Reporter: chunhui shen
>            Assignee: chunhui shen
>         Attachments: HBASE-5918.patch, HBASE-5918.patch
>
>
> When master is initializing, if root server died between assign root and assign meta, master will block at 
> HMaster#assignRootAndMeta:{code}assignmentManager.assignMeta();
> this.catalogTracker.waitForMeta();{code}
> because ServerShutdownHandler is disabled,
> So we should enable ServerShutdownHandler after called assignmentManager.assignMeta();

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5918) Master will block forever at startup if root server dies between assigning root and assigning meta

Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13397583#comment-13397583 ] 

Zhihong Ted Yu commented on HBASE-5918:
---------------------------------------

Patch v2 integrated to trunk.

Thanks for the patch, Chunhui.

Thanks for the review, Stack and Ram.
                
> Master will block forever at startup if root server dies between assigning root and assigning meta
> --------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-5918
>                 URL: https://issues.apache.org/jira/browse/HBASE-5918
>             Project: HBase
>          Issue Type: Bug
>          Components: master
>    Affects Versions: 0.92.1
>            Reporter: chunhui shen
>            Assignee: chunhui shen
>         Attachments: HBASE-5918.patch, HBASE-5918.patch, HBASE-5918V2.patch
>
>
> When master is initializing, if root server died between assign root and assign meta, master will block at 
> HMaster#assignRootAndMeta:{code}assignmentManager.assignMeta();
> this.catalogTracker.waitForMeta();{code}
> because ServerShutdownHandler is disabled,
> So we should enable ServerShutdownHandler after called assignmentManager.assignMeta();

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5918) Master will block forever when startup if root server died between assign root and assign meta

Posted by "stack (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13267187#comment-13267187 ] 

stack commented on HBASE-5918:
------------------------------

Shouldn't we remove the setting of this flag that happens later in finishInitialization?

+1 on the patch otherwise.  This is good stuff.

Any chance of a test?  It looks like it'd be hard to get one in here but it be good if you fellas at least said why a test is hard to squeeze in here to show you at least tried figuring how to test this stuff.  The flag disabling shutdown handler was only added recently but here we find an issue w/ it already.

{code}
Author: Michael Stack <st...@apache.org>  2012-03-13 08:35:54
Committer: Michael Stack <st...@apache.org>  2012-03-13 08:35:54
Parent: fbd4bebd5cca129f49e91ec9936f604998a7025a (HBASE-5314 racefully rolling restart region servers in rolling-restart.sh)
Child:  59e5460807a1dc0fb5763e4b12dda4be49ef3bb4 (HBASE-5574 DEFAULT_MAX_FILE_SIZE defaults to a negative value)
Branches: 094.testfail, 5833trunk, hanging, pbwork, remotes/origin/instant_schema_alter, remotes/origin/trunk, v10, v4, v6
Follows: 
Precedes: 

    HBASE-5179 Handle potential data loss due to concurrent processing of processFaileOver and ServerShutdownHandler
    
    git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1300194 13f79535-47bb-0310-9956-ffa450edef68

----- src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java -----
inde
{code}

                
> Master will block forever when startup if root server died between assign root and assign meta
> ----------------------------------------------------------------------------------------------
>
>                 Key: HBASE-5918
>                 URL: https://issues.apache.org/jira/browse/HBASE-5918
>             Project: HBase
>          Issue Type: Bug
>          Components: master
>    Affects Versions: 0.92.1
>            Reporter: chunhui shen
>            Assignee: chunhui shen
>         Attachments: HBASE-5918.patch, HBASE-5918.patch
>
>
> When master is initializing, if root server died between assign root and assign meta, master will block at 
> HMaster#assignRootAndMeta:{code}assignmentManager.assignMeta();
> this.catalogTracker.waitForMeta();{code}
> because ServerShutdownHandler is disabled,
> So we should enable ServerShutdownHandler after called assignmentManager.assignMeta();

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5918) Master will block forever when startup if root server died between assign root and assign meta

Posted by "chunhui shen (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13267203#comment-13267203 ] 

chunhui shen commented on HBASE-5918:
-------------------------------------

bq.Shouldn't we remove the setting of this flag that happens later in finishInitialization?
In the patch, the setting of this flag is in a if block, so we should keep setting of this flag that happens later in finishInitialization.


                
> Master will block forever when startup if root server died between assign root and assign meta
> ----------------------------------------------------------------------------------------------
>
>                 Key: HBASE-5918
>                 URL: https://issues.apache.org/jira/browse/HBASE-5918
>             Project: HBase
>          Issue Type: Bug
>          Components: master
>    Affects Versions: 0.92.1
>            Reporter: chunhui shen
>            Assignee: chunhui shen
>         Attachments: HBASE-5918.patch, HBASE-5918.patch
>
>
> When master is initializing, if root server died between assign root and assign meta, master will block at 
> HMaster#assignRootAndMeta:{code}assignmentManager.assignMeta();
> this.catalogTracker.waitForMeta();{code}
> because ServerShutdownHandler is disabled,
> So we should enable ServerShutdownHandler after called assignmentManager.assignMeta();

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5918) Master will block forever at startup if root server dies between assigning root and assigning meta

Posted by "Hudson (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13397779#comment-13397779 ] 

Hudson commented on HBASE-5918:
-------------------------------

Integrated in HBase-0.94 #263 (See [https://builds.apache.org/job/HBase-0.94/263/])
    HBASE-5918 Master will block forever at startup if root server dies between assigning root and assigning meta

Submitted by:Chunhui	
Reviewed by:Stack, Ted, Ram (Revision 1352229)

     Result = SUCCESS
ramkrishna : 
Files : 
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/HMaster.java

                
> Master will block forever at startup if root server dies between assigning root and assigning meta
> --------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-5918
>                 URL: https://issues.apache.org/jira/browse/HBASE-5918
>             Project: HBase
>          Issue Type: Bug
>          Components: master
>    Affects Versions: 0.92.1
>            Reporter: chunhui shen
>            Assignee: chunhui shen
>             Fix For: 0.96.0, 0.94.1
>
>         Attachments: HBASE-5918.patch, HBASE-5918.patch, HBASE-5918V2.patch, HBASE-5918_0.94
>
>
> When master is initializing, if root server died between assign root and assign meta, master will block at 
> HMaster#assignRootAndMeta:{code}assignmentManager.assignMeta();
> this.catalogTracker.waitForMeta();{code}
> because ServerShutdownHandler is disabled,
> So we should enable ServerShutdownHandler after called assignmentManager.assignMeta();

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HBASE-5918) Master will block forever when startup if root server died between assign root and assign meta

Posted by "chunhui shen (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-5918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

chunhui shen updated HBASE-5918:
--------------------------------

          Component/s: master
    Affects Version/s: 0.92.1
    
> Master will block forever when startup if root server died between assign root and assign meta
> ----------------------------------------------------------------------------------------------
>
>                 Key: HBASE-5918
>                 URL: https://issues.apache.org/jira/browse/HBASE-5918
>             Project: HBase
>          Issue Type: Bug
>          Components: master
>    Affects Versions: 0.92.1
>            Reporter: chunhui shen
>            Assignee: chunhui shen
>
> When master is initializing, if root server died between assign root and assign meta, master will block at 
> HMaster#assignRootAndMeta:{code}assignmentManager.assignMeta();
> this.catalogTracker.waitForMeta();{code}
> because ServerShutdownHandler is disabled,
> So we should enable ServerShutdownHandler after called assignmentManager.assignMeta();

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5918) Master will block forever when startup if root server died between assign root and assign meta

Posted by "ramkrishna.s.vasudevan (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13397512#comment-13397512 ] 

ramkrishna.s.vasudevan commented on HBASE-5918:
-----------------------------------------------

This issue needs to be committed.  There are few more like this that needs to be integrated I feel.
                
> Master will block forever when startup if root server died between assign root and assign meta
> ----------------------------------------------------------------------------------------------
>
>                 Key: HBASE-5918
>                 URL: https://issues.apache.org/jira/browse/HBASE-5918
>             Project: HBase
>          Issue Type: Bug
>          Components: master
>    Affects Versions: 0.92.1
>            Reporter: chunhui shen
>            Assignee: chunhui shen
>         Attachments: HBASE-5918.patch, HBASE-5918.patch, HBASE-5918V2.patch
>
>
> When master is initializing, if root server died between assign root and assign meta, master will block at 
> HMaster#assignRootAndMeta:{code}assignmentManager.assignMeta();
> this.catalogTracker.waitForMeta();{code}
> because ServerShutdownHandler is disabled,
> So we should enable ServerShutdownHandler after called assignmentManager.assignMeta();

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HBASE-5918) Master will block forever at startup if root server dies between assigning root and assigning meta

Posted by "ramkrishna.s.vasudevan (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-5918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

ramkrishna.s.vasudevan updated HBASE-5918:
------------------------------------------

       Resolution: Fixed
    Fix Version/s: 0.94.1
                   0.96.0
           Status: Resolved  (was: Patch Available)

Resolving after committing it to 0.94. 
                
> Master will block forever at startup if root server dies between assigning root and assigning meta
> --------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-5918
>                 URL: https://issues.apache.org/jira/browse/HBASE-5918
>             Project: HBase
>          Issue Type: Bug
>          Components: master
>    Affects Versions: 0.92.1
>            Reporter: chunhui shen
>            Assignee: chunhui shen
>             Fix For: 0.96.0, 0.94.1
>
>         Attachments: HBASE-5918.patch, HBASE-5918.patch, HBASE-5918V2.patch, HBASE-5918_0.94
>
>
> When master is initializing, if root server died between assign root and assign meta, master will block at 
> HMaster#assignRootAndMeta:{code}assignmentManager.assignMeta();
> this.catalogTracker.waitForMeta();{code}
> because ServerShutdownHandler is disabled,
> So we should enable ServerShutdownHandler after called assignmentManager.assignMeta();

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HBASE-5918) Master will block forever at startup if root server dies between assigning root and assigning meta

Posted by "ramkrishna.s.vasudevan (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-5918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

ramkrishna.s.vasudevan updated HBASE-5918:
------------------------------------------

    Attachment: HBASE-5918_0.94

Patch for 0.94.
                
> Master will block forever at startup if root server dies between assigning root and assigning meta
> --------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-5918
>                 URL: https://issues.apache.org/jira/browse/HBASE-5918
>             Project: HBase
>          Issue Type: Bug
>          Components: master
>    Affects Versions: 0.92.1
>            Reporter: chunhui shen
>            Assignee: chunhui shen
>         Attachments: HBASE-5918.patch, HBASE-5918.patch, HBASE-5918V2.patch, HBASE-5918_0.94
>
>
> When master is initializing, if root server died between assign root and assign meta, master will block at 
> HMaster#assignRootAndMeta:{code}assignmentManager.assignMeta();
> this.catalogTracker.waitForMeta();{code}
> because ServerShutdownHandler is disabled,
> So we should enable ServerShutdownHandler after called assignmentManager.assignMeta();

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5918) Master will block forever at startup if root server dies between assigning root and assigning meta

Posted by "Hudson (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13398015#comment-13398015 ] 

Hudson commented on HBASE-5918:
-------------------------------

Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #61 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/61/])
    HBASE-5918 Master will block forever at startup if root server dies between assigning root and assigning meta (Chunhui) (Revision 1352161)

     Result = FAILURE
tedyu : 
Files : 
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java

                
> Master will block forever at startup if root server dies between assigning root and assigning meta
> --------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-5918
>                 URL: https://issues.apache.org/jira/browse/HBASE-5918
>             Project: HBase
>          Issue Type: Bug
>          Components: master
>    Affects Versions: 0.92.1
>            Reporter: chunhui shen
>            Assignee: chunhui shen
>             Fix For: 0.96.0, 0.94.1
>
>         Attachments: HBASE-5918.patch, HBASE-5918.patch, HBASE-5918V2.patch, HBASE-5918_0.94
>
>
> When master is initializing, if root server died between assign root and assign meta, master will block at 
> HMaster#assignRootAndMeta:{code}assignmentManager.assignMeta();
> this.catalogTracker.waitForMeta();{code}
> because ServerShutdownHandler is disabled,
> So we should enable ServerShutdownHandler after called assignmentManager.assignMeta();

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5918) Master will block forever when startup if root server died between assign root and assign meta

Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13267137#comment-13267137 ] 

Hadoop QA commented on HBASE-5918:
----------------------------------

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12525381/HBASE-5918.patch
  against trunk revision .

    +1 @author.  The patch does not contain any @author tags.

    -1 tests included.  The patch doesn't appear to include any new or modified tests.
                        Please justify why no new tests are needed for this patch.
                        Also please list what manual steps were performed to verify this patch.

    +1 hadoop23.  The patch compiles against the hadoop 0.23.x profile.

    +1 javadoc.  The javadoc tool did not generate any warning messages.

    +1 javac.  The applied patch does not increase the total number of javac compiler warnings.

    +1 findbugs.  The patch does not introduce any new Findbugs (version 1.3.9) warnings.

    +1 release audit.  The applied patch does not increase the total number of release audit warnings.

     -1 core tests.  The patch failed these unit tests:
     

Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1736//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1736//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1736//console

This message is automatically generated.
                
> Master will block forever when startup if root server died between assign root and assign meta
> ----------------------------------------------------------------------------------------------
>
>                 Key: HBASE-5918
>                 URL: https://issues.apache.org/jira/browse/HBASE-5918
>             Project: HBase
>          Issue Type: Bug
>          Components: master
>    Affects Versions: 0.92.1
>            Reporter: chunhui shen
>            Assignee: chunhui shen
>         Attachments: HBASE-5918.patch, HBASE-5918.patch
>
>
> When master is initializing, if root server died between assign root and assign meta, master will block at 
> HMaster#assignRootAndMeta:{code}assignmentManager.assignMeta();
> this.catalogTracker.waitForMeta();{code}
> because ServerShutdownHandler is disabled,
> So we should enable ServerShutdownHandler after called assignmentManager.assignMeta();

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5918) Master will block forever at startup if root server dies between assigning root and assigning meta

Posted by "Hudson (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13397635#comment-13397635 ] 

Hudson commented on HBASE-5918:
-------------------------------

Integrated in HBase-TRUNK #3048 (See [https://builds.apache.org/job/HBase-TRUNK/3048/])
    HBASE-5918 Master will block forever at startup if root server dies between assigning root and assigning meta (Chunhui) (Revision 1352161)

     Result = SUCCESS
tedyu : 
Files : 
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java

                
> Master will block forever at startup if root server dies between assigning root and assigning meta
> --------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-5918
>                 URL: https://issues.apache.org/jira/browse/HBASE-5918
>             Project: HBase
>          Issue Type: Bug
>          Components: master
>    Affects Versions: 0.92.1
>            Reporter: chunhui shen
>            Assignee: chunhui shen
>         Attachments: HBASE-5918.patch, HBASE-5918.patch, HBASE-5918V2.patch
>
>
> When master is initializing, if root server died between assign root and assign meta, master will block at 
> HMaster#assignRootAndMeta:{code}assignmentManager.assignMeta();
> this.catalogTracker.waitForMeta();{code}
> because ServerShutdownHandler is disabled,
> So we should enable ServerShutdownHandler after called assignmentManager.assignMeta();

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HBASE-5918) Master will block forever when startup if root server died between assign root and assign meta

Posted by "chunhui shen (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-5918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

chunhui shen updated HBASE-5918:
--------------------------------

    Attachment: HBASE-5918V2.patch

In the v2 patch, I make a single method that set the flag and then did the call to expireDeadNotExpiredServers. And we will only call this method once now.
                
> Master will block forever when startup if root server died between assign root and assign meta
> ----------------------------------------------------------------------------------------------
>
>                 Key: HBASE-5918
>                 URL: https://issues.apache.org/jira/browse/HBASE-5918
>             Project: HBase
>          Issue Type: Bug
>          Components: master
>    Affects Versions: 0.92.1
>            Reporter: chunhui shen
>            Assignee: chunhui shen
>         Attachments: HBASE-5918.patch, HBASE-5918.patch, HBASE-5918V2.patch
>
>
> When master is initializing, if root server died between assign root and assign meta, master will block at 
> HMaster#assignRootAndMeta:{code}assignmentManager.assignMeta();
> this.catalogTracker.waitForMeta();{code}
> because ServerShutdownHandler is disabled,
> So we should enable ServerShutdownHandler after called assignmentManager.assignMeta();

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HBASE-5918) Master will block forever when startup if root server died between assign root and assign meta

Posted by "chunhui shen (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-5918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

chunhui shen updated HBASE-5918:
--------------------------------

    Attachment: HBASE-5918.patch
    
> Master will block forever when startup if root server died between assign root and assign meta
> ----------------------------------------------------------------------------------------------
>
>                 Key: HBASE-5918
>                 URL: https://issues.apache.org/jira/browse/HBASE-5918
>             Project: HBase
>          Issue Type: Bug
>          Components: master
>    Affects Versions: 0.92.1
>            Reporter: chunhui shen
>            Assignee: chunhui shen
>         Attachments: HBASE-5918.patch
>
>
> When master is initializing, if root server died between assign root and assign meta, master will block at 
> HMaster#assignRootAndMeta:{code}assignmentManager.assignMeta();
> this.catalogTracker.waitForMeta();{code}
> because ServerShutdownHandler is disabled,
> So we should enable ServerShutdownHandler after called assignmentManager.assignMeta();

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5918) Master will block forever when startup if root server died between assign root and assign meta

Posted by "Zhihong Yu (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13267120#comment-13267120 ] 

Zhihong Yu commented on HBASE-5918:
-----------------------------------

@Chunhui:
I used script to analyze https://builds.apache.org/job/PreCommit-HBASE-Build/1720/console but didn't find hanging test.
You may need to run through test suite yourself so that you can find out which test hangs.
                
> Master will block forever when startup if root server died between assign root and assign meta
> ----------------------------------------------------------------------------------------------
>
>                 Key: HBASE-5918
>                 URL: https://issues.apache.org/jira/browse/HBASE-5918
>             Project: HBase
>          Issue Type: Bug
>          Components: master
>    Affects Versions: 0.92.1
>            Reporter: chunhui shen
>            Assignee: chunhui shen
>         Attachments: HBASE-5918.patch, HBASE-5918.patch
>
>
> When master is initializing, if root server died between assign root and assign meta, master will block at 
> HMaster#assignRootAndMeta:{code}assignmentManager.assignMeta();
> this.catalogTracker.waitForMeta();{code}
> because ServerShutdownHandler is disabled,
> So we should enable ServerShutdownHandler after called assignmentManager.assignMeta();

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5918) Master will block forever when startup if root server died between assign root and assign meta

Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13266585#comment-13266585 ] 

Hadoop QA commented on HBASE-5918:
----------------------------------

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12525282/HBASE-5918.patch
  against trunk revision .

    +1 @author.  The patch does not contain any @author tags.

    -1 tests included.  The patch doesn't appear to include any new or modified tests.
                        Please justify why no new tests are needed for this patch.
                        Also please list what manual steps were performed to verify this patch.

    +1 hadoop23.  The patch compiles against the hadoop 0.23.x profile.

    +1 javadoc.  The javadoc tool did not generate any warning messages.

    +1 javac.  The applied patch does not increase the total number of javac compiler warnings.

    -1 findbugs.  The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.

    +1 release audit.  The applied patch does not increase the total number of release audit warnings.

     -1 core tests.  The patch failed these unit tests:
     

Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1720//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1720//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1720//console

This message is automatically generated.
                
> Master will block forever when startup if root server died between assign root and assign meta
> ----------------------------------------------------------------------------------------------
>
>                 Key: HBASE-5918
>                 URL: https://issues.apache.org/jira/browse/HBASE-5918
>             Project: HBase
>          Issue Type: Bug
>          Components: master
>    Affects Versions: 0.92.1
>            Reporter: chunhui shen
>            Assignee: chunhui shen
>         Attachments: HBASE-5918.patch
>
>
> When master is initializing, if root server died between assign root and assign meta, master will block at 
> HMaster#assignRootAndMeta:{code}assignmentManager.assignMeta();
> this.catalogTracker.waitForMeta();{code}
> because ServerShutdownHandler is disabled,
> So we should enable ServerShutdownHandler after called assignmentManager.assignMeta();

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5918) Master will block forever when startup if root server died between assign root and assign meta

Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13267287#comment-13267287 ] 

Hadoop QA commented on HBASE-5918:
----------------------------------

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12525408/HBASE-5918V2.patch
  against trunk revision .

    +1 @author.  The patch does not contain any @author tags.

    -1 tests included.  The patch doesn't appear to include any new or modified tests.
                        Please justify why no new tests are needed for this patch.
                        Also please list what manual steps were performed to verify this patch.

    +1 hadoop23.  The patch compiles against the hadoop 0.23.x profile.

    +1 javadoc.  The javadoc tool did not generate any warning messages.

    +1 javac.  The applied patch does not increase the total number of javac compiler warnings.

    +1 findbugs.  The patch does not introduce any new Findbugs (version 1.3.9) warnings.

    +1 release audit.  The applied patch does not increase the total number of release audit warnings.

    +1 core tests.  The patch passed unit tests in .

Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1742//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1742//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1742//console

This message is automatically generated.
                
> Master will block forever when startup if root server died between assign root and assign meta
> ----------------------------------------------------------------------------------------------
>
>                 Key: HBASE-5918
>                 URL: https://issues.apache.org/jira/browse/HBASE-5918
>             Project: HBase
>          Issue Type: Bug
>          Components: master
>    Affects Versions: 0.92.1
>            Reporter: chunhui shen
>            Assignee: chunhui shen
>         Attachments: HBASE-5918.patch, HBASE-5918.patch, HBASE-5918V2.patch
>
>
> When master is initializing, if root server died between assign root and assign meta, master will block at 
> HMaster#assignRootAndMeta:{code}assignmentManager.assignMeta();
> this.catalogTracker.waitForMeta();{code}
> because ServerShutdownHandler is disabled,
> So we should enable ServerShutdownHandler after called assignmentManager.assignMeta();

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5918) Master will block forever at startup if root server dies between assigning root and assigning meta

Posted by "Hudson (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13400036#comment-13400036 ] 

Hudson commented on HBASE-5918:
-------------------------------

Integrated in HBase-0.94-security #37 (See [https://builds.apache.org/job/HBase-0.94-security/37/])
    HBASE-5918 Master will block forever at startup if root server dies between assigning root and assigning meta

Submitted by:Chunhui	
Reviewed by:Stack, Ted, Ram (Revision 1352229)

     Result = SUCCESS
ramkrishna : 
Files : 
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/HMaster.java

                
> Master will block forever at startup if root server dies between assigning root and assigning meta
> --------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-5918
>                 URL: https://issues.apache.org/jira/browse/HBASE-5918
>             Project: HBase
>          Issue Type: Bug
>          Components: master
>    Affects Versions: 0.92.1
>            Reporter: chunhui shen
>            Assignee: chunhui shen
>             Fix For: 0.96.0, 0.94.1
>
>         Attachments: HBASE-5918.patch, HBASE-5918.patch, HBASE-5918V2.patch, HBASE-5918_0.94
>
>
> When master is initializing, if root server died between assign root and assign meta, master will block at 
> HMaster#assignRootAndMeta:{code}assignmentManager.assignMeta();
> this.catalogTracker.waitForMeta();{code}
> because ServerShutdownHandler is disabled,
> So we should enable ServerShutdownHandler after called assignmentManager.assignMeta();

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HBASE-5918) Master will block forever when startup if root server died between assign root and assign meta

Posted by "chunhui shen (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-5918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

chunhui shen updated HBASE-5918:
--------------------------------

    Attachment: HBASE-5918.patch
    
> Master will block forever when startup if root server died between assign root and assign meta
> ----------------------------------------------------------------------------------------------
>
>                 Key: HBASE-5918
>                 URL: https://issues.apache.org/jira/browse/HBASE-5918
>             Project: HBase
>          Issue Type: Bug
>          Components: master
>    Affects Versions: 0.92.1
>            Reporter: chunhui shen
>            Assignee: chunhui shen
>         Attachments: HBASE-5918.patch, HBASE-5918.patch
>
>
> When master is initializing, if root server died between assign root and assign meta, master will block at 
> HMaster#assignRootAndMeta:{code}assignmentManager.assignMeta();
> this.catalogTracker.waitForMeta();{code}
> because ServerShutdownHandler is disabled,
> So we should enable ServerShutdownHandler after called assignmentManager.assignMeta();

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HBASE-5918) Master will block forever at startup if root server dies between assigning root and assigning meta

Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-5918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Zhihong Ted Yu updated HBASE-5918:
----------------------------------

    Summary: Master will block forever at startup if root server dies between assigning root and assigning meta  (was: Master will block forever when startup if root server died between assign root and assign meta)
    
> Master will block forever at startup if root server dies between assigning root and assigning meta
> --------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-5918
>                 URL: https://issues.apache.org/jira/browse/HBASE-5918
>             Project: HBase
>          Issue Type: Bug
>          Components: master
>    Affects Versions: 0.92.1
>            Reporter: chunhui shen
>            Assignee: chunhui shen
>         Attachments: HBASE-5918.patch, HBASE-5918.patch, HBASE-5918V2.patch
>
>
> When master is initializing, if root server died between assign root and assign meta, master will block at 
> HMaster#assignRootAndMeta:{code}assignmentManager.assignMeta();
> this.catalogTracker.waitForMeta();{code}
> because ServerShutdownHandler is disabled,
> So we should enable ServerShutdownHandler after called assignmentManager.assignMeta();

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Closed] (HBASE-5918) Master will block forever at startup if root server dies between assigning root and assigning meta

Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-5918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Lars Hofhansl closed HBASE-5918.
--------------------------------

    
> Master will block forever at startup if root server dies between assigning root and assigning meta
> --------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-5918
>                 URL: https://issues.apache.org/jira/browse/HBASE-5918
>             Project: HBase
>          Issue Type: Bug
>          Components: master
>    Affects Versions: 0.92.1
>            Reporter: chunhui shen
>            Assignee: chunhui shen
>             Fix For: 0.94.1, 0.96.0
>
>         Attachments: HBASE-5918_0.94, HBASE-5918.patch, HBASE-5918.patch, HBASE-5918V2.patch
>
>
> When master is initializing, if root server died between assign root and assign meta, master will block at 
> HMaster#assignRootAndMeta:{code}assignmentManager.assignMeta();
> this.catalogTracker.waitForMeta();{code}
> because ServerShutdownHandler is disabled,
> So we should enable ServerShutdownHandler after called assignmentManager.assignMeta();

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5918) Master will block forever when startup if root server died between assign root and assign meta

Posted by "Zhihong Yu (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-5918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Zhihong Yu updated HBASE-5918:
------------------------------

    Hadoop Flags: Reviewed
          Status: Patch Available  (was: Open)

Patch makes sense.
                
> Master will block forever when startup if root server died between assign root and assign meta
> ----------------------------------------------------------------------------------------------
>
>                 Key: HBASE-5918
>                 URL: https://issues.apache.org/jira/browse/HBASE-5918
>             Project: HBase
>          Issue Type: Bug
>          Components: master
>    Affects Versions: 0.92.1
>            Reporter: chunhui shen
>            Assignee: chunhui shen
>         Attachments: HBASE-5918.patch
>
>
> When master is initializing, if root server died between assign root and assign meta, master will block at 
> HMaster#assignRootAndMeta:{code}assignmentManager.assignMeta();
> this.catalogTracker.waitForMeta();{code}
> because ServerShutdownHandler is disabled,
> So we should enable ServerShutdownHandler after called assignmentManager.assignMeta();

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5918) Master will block forever when startup if root server died between assign root and assign meta

Posted by "ramkrishna.s.vasudevan (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13267192#comment-13267192 ] 

ramkrishna.s.vasudevan commented on HBASE-5918:
-----------------------------------------------

Even HBASE-5916 is also due to this.  I will try on working on a testcase.
                
> Master will block forever when startup if root server died between assign root and assign meta
> ----------------------------------------------------------------------------------------------
>
>                 Key: HBASE-5918
>                 URL: https://issues.apache.org/jira/browse/HBASE-5918
>             Project: HBase
>          Issue Type: Bug
>          Components: master
>    Affects Versions: 0.92.1
>            Reporter: chunhui shen
>            Assignee: chunhui shen
>         Attachments: HBASE-5918.patch, HBASE-5918.patch
>
>
> When master is initializing, if root server died between assign root and assign meta, master will block at 
> HMaster#assignRootAndMeta:{code}assignmentManager.assignMeta();
> this.catalogTracker.waitForMeta();{code}
> because ServerShutdownHandler is disabled,
> So we should enable ServerShutdownHandler after called assignmentManager.assignMeta();

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira