You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Jean-Daniel Cryans (Created) (JIRA)" <ji...@apache.org> on 2012/01/04 00:36:40 UTC
[jira] [Created] (HBASE-5119) Set the TimeoutMonitor's timeout back
down
Set the TimeoutMonitor's timeout back down
------------------------------------------
Key: HBASE-5119
URL: https://issues.apache.org/jira/browse/HBASE-5119
Project: HBase
Issue Type: Task
Affects Versions: 0.92.0
Reporter: Jean-Daniel Cryans
Priority: Blocker
Fix For: 0.92.0, 0.94.0
The TimeoutMonitor used to be extremely racy and caused more troubles than it fixed, but most of this has been fixed I believe in the context of 0.92 so I think we should set it down back to a useful level. Currently it's 30 minutes, what should the new value be?
I think 5 minutes should be good, will do some testing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5119) Set the TimeoutMonitor's timeout
back down
Posted by "Jean-Daniel Cryans (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13179176#comment-13179176 ]
Jean-Daniel Cryans commented on HBASE-5119:
-------------------------------------------
I think I will retract from my statement that it "used to be extremely racy and caused more troubles than it fixed", on my first test I got a stuck region in transition instead of being able to recover. The timeout was set to 2 minutes to be sure I hit it.
First the region gets closed
{quote}
2012-01-04 00:16:25,811 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Sent CLOSE to sv4r5s38,62023,1325635980913 for region test1,089cd0c9,1325635015491.1a4b111bcc228043e89f59c4c3f6a791.
{quote}
2 minutes later it times out:
{quote}
2012-01-04 00:18:30,026 INFO org.apache.hadoop.hbase.master.AssignmentManager: Regions in transition timed out: test1,089cd0c9,1325635015491.1a4b111bcc228043e89f59c4c3f6a791. state=PENDING_CLOSE, ts=1325636185810, server=null
2012-01-04 00:18:30,026 INFO org.apache.hadoop.hbase.master.AssignmentManager: Region has been PENDING_CLOSE for too long, running forced unassign again on region=test1,089cd0c9,1325635015491.1a4b111bcc228043e89f59c4c3f6a791.
2012-01-04 00:18:30,027 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Starting unassignment of region test1,089cd0c9,1325635015491.1a4b111bcc228043e89f59c4c3f6a791. (offlining)
{quote}
100ms later the master finally gets the event:
{quote}
2012-01-04 00:18:30,129 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Handling transition=RS_ZK_REGION_CLOSED, server=sv4r5s38,62023,1325635980913, region=1a4b111bcc228043e89f59c4c3f6a791, which is more than 15 seconds late
2012-01-04 00:18:30,129 DEBUG org.apache.hadoop.hbase.master.handler.ClosedRegionHandler: Handling CLOSED event for 1a4b111bcc228043e89f59c4c3f6a791
2012-01-04 00:18:30,129 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Table being disabled so deleting ZK node and removing from regions in transition, skipping assignment of region test1,089cd0c9,1325635015491.1a4b111bcc228043e89f59c4c3f6a791.
2012-01-04 00:18:30,129 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: master:62003-0x134589d3db03587 Deleting existing unassigned node for 1a4b111bcc228043e89f59c4c3f6a791 that is in expected state RS_ZK_REGION_CLOSED
2012-01-04 00:18:30,166 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: master:62003-0x134589d3db03587 Successfully deleted unassigned node for region 1a4b111bcc228043e89f59c4c3f6a791 in expected state RS_ZK_REGION_CLOSED
{quote}
At this point everything is fine, the region was processed as closed. But wait, remember that line where it said it was going to force an unassign?
{quote}
2012-01-04 00:18:30,322 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: master:62003-0x134589d3db03587 Creating unassigned node for 1a4b111bcc228043e89f59c4c3f6a791 in a CLOSING state
2012-01-04 00:18:30,328 INFO org.apache.hadoop.hbase.master.AssignmentManager: Server null returned java.lang.NullPointerException: Passed server is null for 1a4b111bcc228043e89f59c4c3f6a791
{quote}
Now the master is confused, it recreated the RIT znode but the region doesn't even exist anymore. It even tries to shut it down but is blocked by NPEs. Now this is what's going on.
The late ZK notification that the znode was deleted (but it got recreated after):
{quote}
2012-01-04 00:19:33,285 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: The znode of region test1,089cd0c9,1325635015491.1a4b111bcc228043e89f59c4c3f6a791. has been deleted.
{quote}
Then it prints this, and much later tries to unassign it again:
{quote}
2012-01-04 00:19:46,607 DEBUG org.apache.hadoop.hbase.master.handler.DeleteTableHandler: Waiting on region to clear regions in transition; test1,089cd0c9,1325635015491.1a4b111bcc228043e89f59c4c3f6a791. state=PENDING_CLOSE, ts=1325636310328, server=null
...
2012-01-04 00:20:39,623 DEBUG org.apache.hadoop.hbase.master.handler.DeleteTableHandler: Waiting on region to clear regions in transition; test1,089cd0c9,1325635015491.1a4b111bcc228043e89f59c4c3f6a791. state=PENDING_CLOSE, ts=1325636310328, server=null
2012-01-04 00:20:39,864 INFO org.apache.hadoop.hbase.master.AssignmentManager: Regions in transition timed out: test1,089cd0c9,1325635015491.1a4b111bcc228043e89f59c4c3f6a791. state=PENDING_CLOSE, ts=1325636310328, server=null
2012-01-04 00:20:39,864 INFO org.apache.hadoop.hbase.master.AssignmentManager: Region has been PENDING_CLOSE for too long, running forced unassign again on region=test1,089cd0c9,1325635015491.1a4b111bcc228043e89f59c4c3f6a791.
2012-01-04 00:20:39,865 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Starting unassignment of region test1,089cd0c9,1325635015491.1a4b111bcc228043e89f59c4c3f6a791. (offlining)
2012-01-04 00:20:39,865 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Attempted to unassign region test1,089cd0c9,1325635015491.1a4b111bcc228043e89f59c4c3f6a791. but it is not currently assigned anywhere
{quote}
And this is still ongoing.
> Set the TimeoutMonitor's timeout back down
> ------------------------------------------
>
> Key: HBASE-5119
> URL: https://issues.apache.org/jira/browse/HBASE-5119
> Project: HBase
> Issue Type: Task
> Affects Versions: 0.92.0
> Reporter: Jean-Daniel Cryans
> Priority: Blocker
> Fix For: 0.92.0, 0.94.0
>
>
> The TimeoutMonitor used to be extremely racy and caused more troubles than it fixed, but most of this has been fixed I believe in the context of 0.92 so I think we should set it down back to a useful level. Currently it's 30 minutes, what should the new value be?
> I think 5 minutes should be good, will do some testing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5119) Set the TimeoutMonitor's timeout back
down
Posted by "Zhihong Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Yu updated HBASE-5119:
------------------------------
Fix Version/s: (was: 0.92.0)
0.92.1
> Set the TimeoutMonitor's timeout back down
> ------------------------------------------
>
> Key: HBASE-5119
> URL: https://issues.apache.org/jira/browse/HBASE-5119
> Project: HBase
> Issue Type: Task
> Affects Versions: 0.92.0
> Reporter: Jean-Daniel Cryans
> Priority: Blocker
> Fix For: 0.94.0, 0.92.1
>
>
> The TimeoutMonitor used to be extremely racy and caused more troubles than it fixed, but most of this has been fixed I believe in the context of 0.92 so I think we should set it down back to a useful level. Currently it's 30 minutes, what should the new value be?
> I think 5 minutes should be good, will do some testing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5119) Set the TimeoutMonitor's timeout
back down
Posted by "Jimmy Xiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jimmy Xiang reassigned HBASE-5119:
----------------------------------
Assignee: (was: Jimmy Xiang)
Based on my testing for HBASE-6611/HBASE-6977, in my testing cluster, I can easily open 10+ regions on 4 nodes in around 4 minutes. I think we are fine to set the timeout back to 5 minutes.
> Set the TimeoutMonitor's timeout back down
> ------------------------------------------
>
> Key: HBASE-5119
> URL: https://issues.apache.org/jira/browse/HBASE-5119
> Project: HBase
> Issue Type: Task
> Affects Versions: 0.92.0
> Reporter: Jean-Daniel Cryans
> Priority: Blocker
> Fix For: 0.96.0
>
>
> The TimeoutMonitor used to be extremely racy and caused more troubles than it fixed, but most of this has been fixed I believe in the context of 0.92 so I think we should set it down back to a useful level. Currently it's 30 minutes, what should the new value be?
> I think 5 minutes should be good, will do some testing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5119) Set the TimeoutMonitor's timeout back
down
Posted by "Lars Hofhansl (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-5119:
---------------------------------
Fix Version/s: (was: 0.92.2)
(was: 0.94.0)
0.96.0
No movement here. Moving to 0.96. Pull back if you feel otherwise.
> Set the TimeoutMonitor's timeout back down
> ------------------------------------------
>
> Key: HBASE-5119
> URL: https://issues.apache.org/jira/browse/HBASE-5119
> Project: HBase
> Issue Type: Task
> Affects Versions: 0.92.0
> Reporter: Jean-Daniel Cryans
> Priority: Blocker
> Fix For: 0.96.0
>
>
> The TimeoutMonitor used to be extremely racy and caused more troubles than it fixed, but most of this has been fixed I believe in the context of 0.92 so I think we should set it down back to a useful level. Currently it's 30 minutes, what should the new value be?
> I think 5 minutes should be good, will do some testing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5119) Set the TimeoutMonitor's timeout
back down
Posted by "Jimmy Xiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13475351#comment-13475351 ]
Jimmy Xiang commented on HBASE-5119:
------------------------------------
I will make the change in HBASE-6611. HBASE-6977 will be helpful too.
> Set the TimeoutMonitor's timeout back down
> ------------------------------------------
>
> Key: HBASE-5119
> URL: https://issues.apache.org/jira/browse/HBASE-5119
> Project: HBase
> Issue Type: Task
> Affects Versions: 0.92.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jimmy Xiang
> Priority: Blocker
> Fix For: 0.96.0
>
>
> The TimeoutMonitor used to be extremely racy and caused more troubles than it fixed, but most of this has been fixed I believe in the context of 0.92 so I think we should set it down back to a useful level. Currently it's 30 minutes, what should the new value be?
> I think 5 minutes should be good, will do some testing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5119) Set the TimeoutMonitor's timeout back
down
Posted by "Jimmy Xiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jimmy Xiang updated HBASE-5119:
-------------------------------
Resolution: Fixed
Release Note: The region assignment timeout time is reduced to 10 minutes. The timeout check interval is reduced to 30 seconds from 60 seconds.
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
> Set the TimeoutMonitor's timeout back down
> ------------------------------------------
>
> Key: HBASE-5119
> URL: https://issues.apache.org/jira/browse/HBASE-5119
> Project: HBase
> Issue Type: Task
> Affects Versions: 0.92.0
> Reporter: Jean-Daniel Cryans
> Priority: Blocker
> Fix For: 0.96.0
>
> Attachments: trunk_5119.patch
>
>
> The TimeoutMonitor used to be extremely racy and caused more troubles than it fixed, but most of this has been fixed I believe in the context of 0.92 so I think we should set it down back to a useful level. Currently it's 30 minutes, what should the new value be?
> I think 5 minutes should be good, will do some testing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5119) Set the TimeoutMonitor's timeout back
down
Posted by "Jimmy Xiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jimmy Xiang updated HBASE-5119:
-------------------------------
Status: Patch Available (was: Open)
> Set the TimeoutMonitor's timeout back down
> ------------------------------------------
>
> Key: HBASE-5119
> URL: https://issues.apache.org/jira/browse/HBASE-5119
> Project: HBase
> Issue Type: Task
> Affects Versions: 0.92.0
> Reporter: Jean-Daniel Cryans
> Priority: Blocker
> Fix For: 0.96.0
>
> Attachments: trunk_5119.patch
>
>
> The TimeoutMonitor used to be extremely racy and caused more troubles than it fixed, but most of this has been fixed I believe in the context of 0.92 so I think we should set it down back to a useful level. Currently it's 30 minutes, what should the new value be?
> I think 5 minutes should be good, will do some testing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5119) Set the TimeoutMonitor's timeout
back down
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13486192#comment-13486192 ]
Hadoop QA commented on HBASE-5119:
----------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12551208/trunk_5119.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 85 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 3 new Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3171//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3171//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3171//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3171//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3171//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3171//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3171//console
This message is automatically generated.
> Set the TimeoutMonitor's timeout back down
> ------------------------------------------
>
> Key: HBASE-5119
> URL: https://issues.apache.org/jira/browse/HBASE-5119
> Project: HBase
> Issue Type: Task
> Affects Versions: 0.92.0
> Reporter: Jean-Daniel Cryans
> Priority: Blocker
> Fix For: 0.96.0
>
> Attachments: trunk_5119.patch
>
>
> The TimeoutMonitor used to be extremely racy and caused more troubles than it fixed, but most of this has been fixed I believe in the context of 0.92 so I think we should set it down back to a useful level. Currently it's 30 minutes, what should the new value be?
> I think 5 minutes should be good, will do some testing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5119) Set the TimeoutMonitor's timeout
back down
Posted by "Jean-Daniel Cryans (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13179203#comment-13179203 ]
Jean-Daniel Cryans commented on HBASE-5119:
-------------------------------------------
In my case killing -9 the master did it.
> Set the TimeoutMonitor's timeout back down
> ------------------------------------------
>
> Key: HBASE-5119
> URL: https://issues.apache.org/jira/browse/HBASE-5119
> Project: HBase
> Issue Type: Task
> Affects Versions: 0.92.0
> Reporter: Jean-Daniel Cryans
> Priority: Blocker
> Fix For: 0.92.0, 0.94.0
>
>
> The TimeoutMonitor used to be extremely racy and caused more troubles than it fixed, but most of this has been fixed I believe in the context of 0.92 so I think we should set it down back to a useful level. Currently it's 30 minutes, what should the new value be?
> I think 5 minutes should be good, will do some testing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5119) Set the TimeoutMonitor's timeout
back down
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13486579#comment-13486579 ]
Hudson commented on HBASE-5119:
-------------------------------
Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #244 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/244/])
HBASE-5119 Set the TimeoutMonitor's timeout back down (Revision 1403500)
Result = FAILURE
jxiang :
Files :
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
> Set the TimeoutMonitor's timeout back down
> ------------------------------------------
>
> Key: HBASE-5119
> URL: https://issues.apache.org/jira/browse/HBASE-5119
> Project: HBase
> Issue Type: Task
> Affects Versions: 0.92.0
> Reporter: Jean-Daniel Cryans
> Priority: Blocker
> Fix For: 0.96.0
>
> Attachments: trunk_5119.patch
>
>
> The TimeoutMonitor used to be extremely racy and caused more troubles than it fixed, but most of this has been fixed I believe in the context of 0.92 so I think we should set it down back to a useful level. Currently it's 30 minutes, what should the new value be?
> I think 5 minutes should be good, will do some testing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5119) Set the TimeoutMonitor's timeout
back down
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13486179#comment-13486179 ]
stack commented on HBASE-5119:
------------------------------
+1 Be sure to make a release note on resolve of the issue detailing new defaults
> Set the TimeoutMonitor's timeout back down
> ------------------------------------------
>
> Key: HBASE-5119
> URL: https://issues.apache.org/jira/browse/HBASE-5119
> Project: HBase
> Issue Type: Task
> Affects Versions: 0.92.0
> Reporter: Jean-Daniel Cryans
> Priority: Blocker
> Fix For: 0.96.0
>
> Attachments: trunk_5119.patch
>
>
> The TimeoutMonitor used to be extremely racy and caused more troubles than it fixed, but most of this has been fixed I believe in the context of 0.92 so I think we should set it down back to a useful level. Currently it's 30 minutes, what should the new value be?
> I think 5 minutes should be good, will do some testing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5119) Set the TimeoutMonitor's timeout
back down
Posted by "Jimmy Xiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13486137#comment-13486137 ]
Jimmy Xiang commented on HBASE-5119:
------------------------------------
I uploaded a patch, which simply set the timeout to 10minutes and the checking period to 30 seconds.
> Set the TimeoutMonitor's timeout back down
> ------------------------------------------
>
> Key: HBASE-5119
> URL: https://issues.apache.org/jira/browse/HBASE-5119
> Project: HBase
> Issue Type: Task
> Affects Versions: 0.92.0
> Reporter: Jean-Daniel Cryans
> Priority: Blocker
> Fix For: 0.96.0
>
> Attachments: trunk_5119.patch
>
>
> The TimeoutMonitor used to be extremely racy and caused more troubles than it fixed, but most of this has been fixed I believe in the context of 0.92 so I think we should set it down back to a useful level. Currently it's 30 minutes, what should the new value be?
> I think 5 minutes should be good, will do some testing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5119) Set the TimeoutMonitor's timeout back
down
Posted by "stack (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-5119:
-------------------------
Fix Version/s: (was: 0.92.1)
0.92.2
Moving to 0.92.2
> Set the TimeoutMonitor's timeout back down
> ------------------------------------------
>
> Key: HBASE-5119
> URL: https://issues.apache.org/jira/browse/HBASE-5119
> Project: HBase
> Issue Type: Task
> Affects Versions: 0.92.0
> Reporter: Jean-Daniel Cryans
> Priority: Blocker
> Fix For: 0.94.0, 0.92.2
>
>
> The TimeoutMonitor used to be extremely racy and caused more troubles than it fixed, but most of this has been fixed I believe in the context of 0.92 so I think we should set it down back to a useful level. Currently it's 30 minutes, what should the new value be?
> I think 5 minutes should be good, will do some testing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (HBASE-5119) Set the TimeoutMonitor's
timeout back down
Posted by "Jimmy Xiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13481975#comment-13481975 ]
Jimmy Xiang edited comment on HBASE-5119 at 10/29/12 4:45 PM:
--------------------------------------------------------------
Based on my testing for HBASE-6611/HBASE-6977, in my testing cluster, I can easily open 10+k regions on 4 nodes in around 4 minutes. I think we are fine to set the timeout back to 5 minutes.
was (Author: jxiang):
Based on my testing for HBASE-6611/HBASE-6977, in my testing cluster, I can easily open 10+ regions on 4 nodes in around 4 minutes. I think we are fine to set the timeout back to 5 minutes.
> Set the TimeoutMonitor's timeout back down
> ------------------------------------------
>
> Key: HBASE-5119
> URL: https://issues.apache.org/jira/browse/HBASE-5119
> Project: HBase
> Issue Type: Task
> Affects Versions: 0.92.0
> Reporter: Jean-Daniel Cryans
> Priority: Blocker
> Fix For: 0.96.0
>
> Attachments: trunk_5119.patch
>
>
> The TimeoutMonitor used to be extremely racy and caused more troubles than it fixed, but most of this has been fixed I believe in the context of 0.92 so I think we should set it down back to a useful level. Currently it's 30 minutes, what should the new value be?
> I think 5 minutes should be good, will do some testing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5119) Set the TimeoutMonitor's timeout back
down
Posted by "Jimmy Xiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jimmy Xiang updated HBASE-5119:
-------------------------------
Attachment: trunk_5119.patch
> Set the TimeoutMonitor's timeout back down
> ------------------------------------------
>
> Key: HBASE-5119
> URL: https://issues.apache.org/jira/browse/HBASE-5119
> Project: HBase
> Issue Type: Task
> Affects Versions: 0.92.0
> Reporter: Jean-Daniel Cryans
> Priority: Blocker
> Fix For: 0.96.0
>
> Attachments: trunk_5119.patch
>
>
> The TimeoutMonitor used to be extremely racy and caused more troubles than it fixed, but most of this has been fixed I believe in the context of 0.92 so I think we should set it down back to a useful level. Currently it's 30 minutes, what should the new value be?
> I think 5 minutes should be good, will do some testing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5119) Set the TimeoutMonitor's timeout
back down
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13179193#comment-13179193 ]
Lars Hofhansl commented on HBASE-5119:
--------------------------------------
This stuff scares me. How do you manually recover from this?
> Set the TimeoutMonitor's timeout back down
> ------------------------------------------
>
> Key: HBASE-5119
> URL: https://issues.apache.org/jira/browse/HBASE-5119
> Project: HBase
> Issue Type: Task
> Affects Versions: 0.92.0
> Reporter: Jean-Daniel Cryans
> Priority: Blocker
> Fix For: 0.92.0, 0.94.0
>
>
> The TimeoutMonitor used to be extremely racy and caused more troubles than it fixed, but most of this has been fixed I believe in the context of 0.92 so I think we should set it down back to a useful level. Currently it's 30 minutes, what should the new value be?
> I think 5 minutes should be good, will do some testing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5119) Set the TimeoutMonitor's timeout
back down
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13486415#comment-13486415 ]
Hudson commented on HBASE-5119:
-------------------------------
Integrated in HBase-TRUNK #3495 (See [https://builds.apache.org/job/HBase-TRUNK/3495/])
HBASE-5119 Set the TimeoutMonitor's timeout back down (Revision 1403500)
Result = SUCCESS
jxiang :
Files :
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
> Set the TimeoutMonitor's timeout back down
> ------------------------------------------
>
> Key: HBASE-5119
> URL: https://issues.apache.org/jira/browse/HBASE-5119
> Project: HBase
> Issue Type: Task
> Affects Versions: 0.92.0
> Reporter: Jean-Daniel Cryans
> Priority: Blocker
> Fix For: 0.96.0
>
> Attachments: trunk_5119.patch
>
>
> The TimeoutMonitor used to be extremely racy and caused more troubles than it fixed, but most of this has been fixed I believe in the context of 0.92 so I think we should set it down back to a useful level. Currently it's 30 minutes, what should the new value be?
> I think 5 minutes should be good, will do some testing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5119) Set the TimeoutMonitor's timeout
back down
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13485555#comment-13485555 ]
stack commented on HBASE-5119:
------------------------------
Make a patch to set it to 10 minutes Jimmy? That would be a big improvement over what it currently is?
> Set the TimeoutMonitor's timeout back down
> ------------------------------------------
>
> Key: HBASE-5119
> URL: https://issues.apache.org/jira/browse/HBASE-5119
> Project: HBase
> Issue Type: Task
> Affects Versions: 0.92.0
> Reporter: Jean-Daniel Cryans
> Priority: Blocker
> Fix For: 0.96.0
>
>
> The TimeoutMonitor used to be extremely racy and caused more troubles than it fixed, but most of this has been fixed I believe in the context of 0.92 so I think we should set it down back to a useful level. Currently it's 30 minutes, what should the new value be?
> I think 5 minutes should be good, will do some testing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5119) Set the TimeoutMonitor's timeout
back down
Posted by "Jimmy Xiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13475350#comment-13475350 ]
Jimmy Xiang commented on HBASE-5119:
------------------------------------
We can do it in 0.96.
> Set the TimeoutMonitor's timeout back down
> ------------------------------------------
>
> Key: HBASE-5119
> URL: https://issues.apache.org/jira/browse/HBASE-5119
> Project: HBase
> Issue Type: Task
> Affects Versions: 0.92.0
> Reporter: Jean-Daniel Cryans
> Priority: Blocker
> Fix For: 0.96.0
>
>
> The TimeoutMonitor used to be extremely racy and caused more troubles than it fixed, but most of this has been fixed I believe in the context of 0.92 so I think we should set it down back to a useful level. Currently it's 30 minutes, what should the new value be?
> I think 5 minutes should be good, will do some testing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5119) Set the TimeoutMonitor's timeout
back down
Posted by "Jimmy Xiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jimmy Xiang reassigned HBASE-5119:
----------------------------------
Assignee: Jimmy Xiang
> Set the TimeoutMonitor's timeout back down
> ------------------------------------------
>
> Key: HBASE-5119
> URL: https://issues.apache.org/jira/browse/HBASE-5119
> Project: HBase
> Issue Type: Task
> Affects Versions: 0.92.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jimmy Xiang
> Priority: Blocker
> Fix For: 0.96.0
>
>
> The TimeoutMonitor used to be extremely racy and caused more troubles than it fixed, but most of this has been fixed I believe in the context of 0.92 so I think we should set it down back to a useful level. Currently it's 30 minutes, what should the new value be?
> I think 5 minutes should be good, will do some testing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira