You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "Enis Soztutar (JIRA)" <ji...@apache.org> on 2015/09/24 02:54:04 UTC

[jira] [Created] (HBASE-14474) DeadLock in RpcClientImpl.Connection.close()

Enis Soztutar created HBASE-14474:
-------------------------------------

             Summary: DeadLock in RpcClientImpl.Connection.close() 
                 Key: HBASE-14474
                 URL: https://issues.apache.org/jira/browse/HBASE-14474
             Project: HBase
          Issue Type: Bug
          Components: rpc
            Reporter: Enis Soztutar
            Assignee: Enis Soztutar
            Priority: Blocker
             Fix For: 2.0.0, 1.2.0, 1.0.3, 1.1.3


>From a code base that contains 1.1.2 + HBASE-14449 + HBASE-14241 and HBASE-14313 we can reproduce a dead lock with serverKilling CM easily: 

{code}
Found one Java-level deadlock: ============================= "htable-pool1-t63":   waiting to lock monitor 0x0000000001cb1688 (object 0x00000000806ef150, a org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection),   which is held by "IPC Client (1403704789) connection to enis-hbase-sep-21-6.novalocal/172.22.107.106:16020 from root" "IPC Client (1403704789) connection to enis-hbase-sep-21-6.novalocal/172.22.107.106:16020 from root":   waiting to lock monitor 0x0000000001cb1738 (object 0x00000000806f0c60, a java.lang.Object),   which is held by "htable-pool1-t63" Java stack information for the threads listed above: =================================================== "htable-pool1-t63":         at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.close(RpcClientImpl.java:819)         - waiting to lock <0x00000000806ef150> (a org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection)         at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:906)         - locked <0x00000000806f0c60> (a java.lang.Object)         at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:856)         at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1192)         at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:213)         at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:287)         at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.multi(ClientProtos.java:32699)         at org.apache.hadoop.hbase.client.MultiServerCallable.call(MultiServerCallable.java:129)         at org.apache.hadoop.hbase.client.MultiServerCallable.call(MultiServerCallable.java:54)         at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:200)         at org.apache.hadoop.hbase.client.AsyncProcess$AsyncRequestFutureImpl$SingleServerRequestRunnable.run(AsyncProcess.java:708)         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)         at java.util.concurrent.FutureTask.run(FutureTask.java:266)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745) "IPC Client (1403704789) connection to enis-hbase-sep-21-6.novalocal/172.22.107.106:16020 from root":         at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.close(RpcClientImpl.java:832)         - waiting to lock <0x00000000806f0c60> (a java.lang.Object)         - locked <0x00000000806ef150> (a org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection)         at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.run(RpcClientImpl.java:574)
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)