You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org> on 2013/08/23 20:11:52 UTC
[jira] [Created] (HBASE-9322) Contention on the SoftValueSortedMap
with multiple clients
Jean-Daniel Cryans created HBASE-9322:
-----------------------------------------
Summary: Contention on the SoftValueSortedMap with multiple clients
Key: HBASE-9322
URL: https://issues.apache.org/jira/browse/HBASE-9322
Project: HBase
Issue Type: Bug
Affects Versions: 0.95.2
Reporter: Jean-Daniel Cryans
Fix For: 0.98.0, 0.96.0
I've been running tests on clusters with "lots" of regions, about 400, and I'm seeing weird contention in the client.
This one shows up a lot around the SoftValueSortedMap.
First I have this blocked thread on I'm not sure what:
{noformat}
"TestClient-12" prio=10 tid=0x00007fb268872000 nid=0x3add waiting for monitor entry [0x00007fb251416000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.apache.hadoop.hbase.util.SoftValueSortedMap.isEmpty(SoftValueSortedMap.java:210)
- locked <0x00000000c1b70318> (a java.util.TreeMap)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getCachedLocation(HConnectionManager.java:1263)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1103)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:1036)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:993)
{noformat}
This client waits on it:
{noformat}
"TestClient-14" prio=10 tid=0x00007fb268876000 nid=0x3adf waiting for monitor entry [0x00007fb251214000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.apache.hadoop.hbase.util.SoftValueSortedMap.lowerValueByKey(SoftValueSortedMap.java:189)
- waiting to lock <0x00000000c1b70318> (a java.util.TreeMap)
- locked <0x00000000c1b82120> (a org.apache.hadoop.hbase.util.SoftValueSortedMap)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getCachedLocation(HConnectionManager.java:1272)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1103)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:1036)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:993)
{noformat}
As you can see it's also holding a lock, which I have 11 other clients waiting on:
{noformat}
"TestClient-13" prio=10 tid=0x00007fb268874000 nid=0x3ade waiting for monitor entry [0x00007fb251315000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.apache.hadoop.hbase.util.SoftValueSortedMap.lowerValueByKey(SoftValueSortedMap.java:189)
- waiting to lock <0x00000000c1b82120> (a org.apache.hadoop.hbase.util.SoftValueSortedMap)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getCachedLocation(HConnectionManager.java:1272)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1103)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:1036)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:993)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira