You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Rajeshbabu Chintaguntla (JIRA)" <ji...@apache.org> on 2015/01/22 03:13:35 UTC
[jira] [Created] (HBASE-12901) Possible deadlock while onlining a
region and get region plan for other region run parallel
Rajeshbabu Chintaguntla created HBASE-12901:
-----------------------------------------------
Summary: Possible deadlock while onlining a region and get region plan for other region run parallel
Key: HBASE-12901
URL: https://issues.apache.org/jira/browse/HBASE-12901
Project: HBase
Issue Type: Bug
Reporter: Rajeshbabu Chintaguntla
Assignee: Rajeshbabu Chintaguntla
Priority: Critical
Fix For: 1.0.0, 1.1.0
There is a deadlock when region state updating(regionOnline)after assignment completed and getting region plan to other region parallelly. Before onlining we are synchronizing on regionStates and inside synchronizing on regionPlans to clear the region plan. At the same time there is a chance that while getting plan first we synchornize on regionPlans and then regionStates while getting assignments of a server. This is coming after HBASE-12686 fix. This issue present in branch-1 and branch-1.1 only.
{code}
"AM.-pool1-t33":
at org.apache.hadoop.hbase.master.AssignmentManager.clearRegionPlan(AssignmentManager.java:2917)
- waiting to lock <0x00000000d0147f70> (a java.util.TreeMap)
at org.apache.hadoop.hbase.master.AssignmentManager.regionOffline(AssignmentManager.java:3617)
at org.apache.hadoop.hbase.master.AssignmentManager.regionOffline(AssignmentManager.java:1402)
at org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1734)
at org.apache.hadoop.hbase.master.AssignmentManager.forceRegionStateToOffline(AssignmentManager.java:1821)
at org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1456)
at org.apache.hadoop.hbase.master.AssignCallable.call(AssignCallable.java:45)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
"AM.-pool1-t29":
at org.apache.hadoop.hbase.master.RegionStates.getRegionAssignments(RegionStates.java:155)
- waiting to lock <0x00000000d010b250> (a org.apache.hadoop.hbase.master.RegionStates)
at org.apache.hadoop.hbase.master.AssignmentManager.getSnapShotOfAssignment(AssignmentManager.java:3629)
at org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.getRegionAssignmentsByServer(BaseLoadBalancer.java:1146)
at org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.createCluster(BaseLoadBalancer.java:959)
at org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.randomAssignment(BaseLoadBalancer.java:1010)
at org.apache.hadoop.hbase.master.AssignmentManager.getRegionPlan(AssignmentManager.java:2228)
- locked <0x00000000d0147f70> (a java.util.TreeMap)
at org.apache.hadoop.hbase.master.AssignmentManager.getRegionPlan(AssignmentManager.java:2185)
at org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1905)
at org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1464)
at org.apache.hadoop.hbase.master.AssignCallable.call(AssignCallable.java:45)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
"AM.ZK.Worker-pool2-t41":
at org.apache.hadoop.hbase.master.AssignmentManager.clearRegionPlan(AssignmentManager.java:2917)
- waiting to lock <0x00000000d0147f70> (a java.util.TreeMap)
at org.apache.hadoop.hbase.master.AssignmentManager.regionOnline(AssignmentManager.java:1305)
at org.apache.hadoop.hbase.master.AssignmentManager$4.run(AssignmentManager.java:1196)
- locked <0x00000000d010b250> (a org.apache.hadoop.hbase.master.RegionStates)
at org.apache.hadoop.hbase.master.AssignmentManager$3.run(AssignmentManager.java:1142)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)