You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Jean-Daniel Cryans (Created) (JIRA)" <ji...@apache.org> on 2011/11/02 01:43:32 UTC
[jira] [Created] (HBASE-4723) Loads of
NotAllMetaRegionsOnlineException traces when starting the master
Loads of NotAllMetaRegionsOnlineException traces when starting the master
-------------------------------------------------------------------------
Key: HBASE-4723
URL: https://issues.apache.org/jira/browse/HBASE-4723
Project: HBase
Issue Type: Improvement
Reporter: Jean-Daniel Cryans
Assignee: Jean-Daniel Cryans
Priority: Minor
Fix For: 0.92.0, 0.90.5
Minor annoyance, when starting a master I very often get 1 or more stack traces like these:
{quote}
2011-11-02 00:39:14,448 INFO org.apache.hadoop.hbase.catalog.CatalogTracker: Retrying
org.apache.hadoop.hbase.NotAllMetaRegionsOnlineException: Timed out (100ms)
at org.apache.hadoop.hbase.catalog.CatalogTracker.waitForMeta(CatalogTracker.java:449)
at org.apache.hadoop.hbase.catalog.CatalogTracker.waitForMeta(CatalogTracker.java:413)
at org.apache.hadoop.hbase.master.HMaster.assignRootAndMeta(HMaster.java:541)
at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:468)
at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:309)
at java.lang.Thread.run(Thread.java:662)
{quote}
1) it's not super clear what's going on (putting myself in a new user's head) and 2) those exceptions look bad (until you see they are at INFO level).
I'd just do a little cleanup: remove the stack trace, add a more meaningful message.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4723) Loads of
NotAllMetaRegionsOnlineException traces when starting the master
Posted by "stack (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-4723:
-------------------------
Resolution: Fixed
Fix Version/s: (was: 0.90.5)
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Applied branch and 0.92 trunk.
> Loads of NotAllMetaRegionsOnlineException traces when starting the master
> -------------------------------------------------------------------------
>
> Key: HBASE-4723
> URL: https://issues.apache.org/jira/browse/HBASE-4723
> Project: HBase
> Issue Type: Improvement
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Priority: Minor
> Fix For: 0.92.0
>
> Attachments: 4723-v2.txt, HBASE-4723.patch
>
>
> Minor annoyance, when starting a master I very often get 1 or more stack traces like these:
> {quote}
> 2011-11-02 00:39:14,448 INFO org.apache.hadoop.hbase.catalog.CatalogTracker: Retrying
> org.apache.hadoop.hbase.NotAllMetaRegionsOnlineException: Timed out (100ms)
> at org.apache.hadoop.hbase.catalog.CatalogTracker.waitForMeta(CatalogTracker.java:449)
> at org.apache.hadoop.hbase.catalog.CatalogTracker.waitForMeta(CatalogTracker.java:413)
> at org.apache.hadoop.hbase.master.HMaster.assignRootAndMeta(HMaster.java:541)
> at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:468)
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:309)
> at java.lang.Thread.run(Thread.java:662)
> {quote}
> 1) it's not super clear what's going on (putting myself in a new user's head) and 2) those exceptions look bad (until you see they are at INFO level).
> I'd just do a little cleanup: remove the stack trace, add a more meaningful message.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4723) Loads of
NotAllMetaRegionsOnlineException traces when starting the master
Posted by "stack (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-4723:
-------------------------
Attachment: 4723-v2.txt
Here is what I applied. Patch wouldn't apply.
> Loads of NotAllMetaRegionsOnlineException traces when starting the master
> -------------------------------------------------------------------------
>
> Key: HBASE-4723
> URL: https://issues.apache.org/jira/browse/HBASE-4723
> Project: HBase
> Issue Type: Improvement
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Priority: Minor
> Fix For: 0.92.0
>
> Attachments: 4723-v2.txt, HBASE-4723.patch
>
>
> Minor annoyance, when starting a master I very often get 1 or more stack traces like these:
> {quote}
> 2011-11-02 00:39:14,448 INFO org.apache.hadoop.hbase.catalog.CatalogTracker: Retrying
> org.apache.hadoop.hbase.NotAllMetaRegionsOnlineException: Timed out (100ms)
> at org.apache.hadoop.hbase.catalog.CatalogTracker.waitForMeta(CatalogTracker.java:449)
> at org.apache.hadoop.hbase.catalog.CatalogTracker.waitForMeta(CatalogTracker.java:413)
> at org.apache.hadoop.hbase.master.HMaster.assignRootAndMeta(HMaster.java:541)
> at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:468)
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:309)
> at java.lang.Thread.run(Thread.java:662)
> {quote}
> 1) it's not super clear what's going on (putting myself in a new user's head) and 2) those exceptions look bad (until you see they are at INFO level).
> I'd just do a little cleanup: remove the stack trace, add a more meaningful message.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4723) Loads of
NotAllMetaRegionsOnlineException traces when starting the master
Posted by "Jean-Daniel Cryans (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jean-Daniel Cryans updated HBASE-4723:
--------------------------------------
Status: Patch Available (was: Open)
> Loads of NotAllMetaRegionsOnlineException traces when starting the master
> -------------------------------------------------------------------------
>
> Key: HBASE-4723
> URL: https://issues.apache.org/jira/browse/HBASE-4723
> Project: HBase
> Issue Type: Improvement
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Priority: Minor
> Fix For: 0.92.0, 0.90.5
>
> Attachments: HBASE-4723.patch
>
>
> Minor annoyance, when starting a master I very often get 1 or more stack traces like these:
> {quote}
> 2011-11-02 00:39:14,448 INFO org.apache.hadoop.hbase.catalog.CatalogTracker: Retrying
> org.apache.hadoop.hbase.NotAllMetaRegionsOnlineException: Timed out (100ms)
> at org.apache.hadoop.hbase.catalog.CatalogTracker.waitForMeta(CatalogTracker.java:449)
> at org.apache.hadoop.hbase.catalog.CatalogTracker.waitForMeta(CatalogTracker.java:413)
> at org.apache.hadoop.hbase.master.HMaster.assignRootAndMeta(HMaster.java:541)
> at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:468)
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:309)
> at java.lang.Thread.run(Thread.java:662)
> {quote}
> 1) it's not super clear what's going on (putting myself in a new user's head) and 2) those exceptions look bad (until you see they are at INFO level).
> I'd just do a little cleanup: remove the stack trace, add a more meaningful message.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4723) Loads of
NotAllMetaRegionsOnlineException traces when starting the master
Posted by "Jean-Daniel Cryans (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jean-Daniel Cryans updated HBASE-4723:
--------------------------------------
Attachment: HBASE-4723.patch
Extremely simple patch, it looks like this when running:
bq. 2011-11-02 00:50:55,981 INFO org.apache.hadoop.hbase.catalog.CatalogTracker: .META. still not available, sleeping and retrying. Reason: Timed out (100ms)
> Loads of NotAllMetaRegionsOnlineException traces when starting the master
> -------------------------------------------------------------------------
>
> Key: HBASE-4723
> URL: https://issues.apache.org/jira/browse/HBASE-4723
> Project: HBase
> Issue Type: Improvement
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Priority: Minor
> Fix For: 0.92.0, 0.90.5
>
> Attachments: HBASE-4723.patch
>
>
> Minor annoyance, when starting a master I very often get 1 or more stack traces like these:
> {quote}
> 2011-11-02 00:39:14,448 INFO org.apache.hadoop.hbase.catalog.CatalogTracker: Retrying
> org.apache.hadoop.hbase.NotAllMetaRegionsOnlineException: Timed out (100ms)
> at org.apache.hadoop.hbase.catalog.CatalogTracker.waitForMeta(CatalogTracker.java:449)
> at org.apache.hadoop.hbase.catalog.CatalogTracker.waitForMeta(CatalogTracker.java:413)
> at org.apache.hadoop.hbase.master.HMaster.assignRootAndMeta(HMaster.java:541)
> at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:468)
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:309)
> at java.lang.Thread.run(Thread.java:662)
> {quote}
> 1) it's not super clear what's going on (putting myself in a new user's head) and 2) those exceptions look bad (until you see they are at INFO level).
> I'd just do a little cleanup: remove the stack trace, add a more meaningful message.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4723) Loads of
NotAllMetaRegionsOnlineException traces when starting the master
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13148979#comment-13148979 ]
Hudson commented on HBASE-4723:
-------------------------------
Integrated in HBase-TRUNK #2430 (See [https://builds.apache.org/job/HBase-TRUNK/2430/])
HBASE-4723 Loads of NotAllMetaRegionsOnlineException traces when starting the master
stack :
Files :
* /hbase/trunk/CHANGES.txt
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java
> Loads of NotAllMetaRegionsOnlineException traces when starting the master
> -------------------------------------------------------------------------
>
> Key: HBASE-4723
> URL: https://issues.apache.org/jira/browse/HBASE-4723
> Project: HBase
> Issue Type: Improvement
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Priority: Minor
> Fix For: 0.92.0
>
> Attachments: 4723-v2.txt, HBASE-4723.patch
>
>
> Minor annoyance, when starting a master I very often get 1 or more stack traces like these:
> {quote}
> 2011-11-02 00:39:14,448 INFO org.apache.hadoop.hbase.catalog.CatalogTracker: Retrying
> org.apache.hadoop.hbase.NotAllMetaRegionsOnlineException: Timed out (100ms)
> at org.apache.hadoop.hbase.catalog.CatalogTracker.waitForMeta(CatalogTracker.java:449)
> at org.apache.hadoop.hbase.catalog.CatalogTracker.waitForMeta(CatalogTracker.java:413)
> at org.apache.hadoop.hbase.master.HMaster.assignRootAndMeta(HMaster.java:541)
> at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:468)
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:309)
> at java.lang.Thread.run(Thread.java:662)
> {quote}
> 1) it's not super clear what's going on (putting myself in a new user's head) and 2) those exceptions look bad (until you see they are at INFO level).
> I'd just do a little cleanup: remove the stack trace, add a more meaningful message.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4723) Loads of
NotAllMetaRegionsOnlineException traces when starting the master
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13141898#comment-13141898 ]
stack commented on HBASE-4723:
------------------------------
+1 but change log message on commit because it could be -ROOT- that is not yet on-line; its not just .META. issue.
The TestDistributedLogSplitting failed because of 'too many open files'. I added logging ulimit and host name to the build config so we can see which machine is w/o the right ulimit setting (Because Giri today said he'd set them all to 16k). Lets see.
> Loads of NotAllMetaRegionsOnlineException traces when starting the master
> -------------------------------------------------------------------------
>
> Key: HBASE-4723
> URL: https://issues.apache.org/jira/browse/HBASE-4723
> Project: HBase
> Issue Type: Improvement
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Priority: Minor
> Fix For: 0.92.0, 0.90.5
>
> Attachments: HBASE-4723.patch
>
>
> Minor annoyance, when starting a master I very often get 1 or more stack traces like these:
> {quote}
> 2011-11-02 00:39:14,448 INFO org.apache.hadoop.hbase.catalog.CatalogTracker: Retrying
> org.apache.hadoop.hbase.NotAllMetaRegionsOnlineException: Timed out (100ms)
> at org.apache.hadoop.hbase.catalog.CatalogTracker.waitForMeta(CatalogTracker.java:449)
> at org.apache.hadoop.hbase.catalog.CatalogTracker.waitForMeta(CatalogTracker.java:413)
> at org.apache.hadoop.hbase.master.HMaster.assignRootAndMeta(HMaster.java:541)
> at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:468)
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:309)
> at java.lang.Thread.run(Thread.java:662)
> {quote}
> 1) it's not super clear what's going on (putting myself in a new user's head) and 2) those exceptions look bad (until you see they are at INFO level).
> I'd just do a little cleanup: remove the stack trace, add a more meaningful message.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4723) Loads of
NotAllMetaRegionsOnlineException traces when starting the master
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13141861#comment-13141861 ]
Hadoop QA commented on HBASE-4723:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12501885/HBASE-4723.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
-1 javadoc. The javadoc tool appears to have generated -165 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 2 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestDistributedLogSplitting
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/132//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/132//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/132//console
This message is automatically generated.
> Loads of NotAllMetaRegionsOnlineException traces when starting the master
> -------------------------------------------------------------------------
>
> Key: HBASE-4723
> URL: https://issues.apache.org/jira/browse/HBASE-4723
> Project: HBase
> Issue Type: Improvement
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Priority: Minor
> Fix For: 0.92.0, 0.90.5
>
> Attachments: HBASE-4723.patch
>
>
> Minor annoyance, when starting a master I very often get 1 or more stack traces like these:
> {quote}
> 2011-11-02 00:39:14,448 INFO org.apache.hadoop.hbase.catalog.CatalogTracker: Retrying
> org.apache.hadoop.hbase.NotAllMetaRegionsOnlineException: Timed out (100ms)
> at org.apache.hadoop.hbase.catalog.CatalogTracker.waitForMeta(CatalogTracker.java:449)
> at org.apache.hadoop.hbase.catalog.CatalogTracker.waitForMeta(CatalogTracker.java:413)
> at org.apache.hadoop.hbase.master.HMaster.assignRootAndMeta(HMaster.java:541)
> at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:468)
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:309)
> at java.lang.Thread.run(Thread.java:662)
> {quote}
> 1) it's not super clear what's going on (putting myself in a new user's head) and 2) those exceptions look bad (until you see they are at INFO level).
> I'd just do a little cleanup: remove the stack trace, add a more meaningful message.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4723) Loads of
NotAllMetaRegionsOnlineException traces when starting the master
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13148947#comment-13148947 ]
Hudson commented on HBASE-4723:
-------------------------------
Integrated in HBase-0.92 #125 (See [https://builds.apache.org/job/HBase-0.92/125/])
HBASE-4723 Loads of NotAllMetaRegionsOnlineException traces when starting the master
stack :
Files :
* /hbase/branches/0.92/CHANGES.txt
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java
> Loads of NotAllMetaRegionsOnlineException traces when starting the master
> -------------------------------------------------------------------------
>
> Key: HBASE-4723
> URL: https://issues.apache.org/jira/browse/HBASE-4723
> Project: HBase
> Issue Type: Improvement
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Priority: Minor
> Fix For: 0.92.0
>
> Attachments: 4723-v2.txt, HBASE-4723.patch
>
>
> Minor annoyance, when starting a master I very often get 1 or more stack traces like these:
> {quote}
> 2011-11-02 00:39:14,448 INFO org.apache.hadoop.hbase.catalog.CatalogTracker: Retrying
> org.apache.hadoop.hbase.NotAllMetaRegionsOnlineException: Timed out (100ms)
> at org.apache.hadoop.hbase.catalog.CatalogTracker.waitForMeta(CatalogTracker.java:449)
> at org.apache.hadoop.hbase.catalog.CatalogTracker.waitForMeta(CatalogTracker.java:413)
> at org.apache.hadoop.hbase.master.HMaster.assignRootAndMeta(HMaster.java:541)
> at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:468)
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:309)
> at java.lang.Thread.run(Thread.java:662)
> {quote}
> 1) it's not super clear what's going on (putting myself in a new user's head) and 2) those exceptions look bad (until you see they are at INFO level).
> I'd just do a little cleanup: remove the stack trace, add a more meaningful message.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira