HBase client shutdown hook hangs

[Henning Blohm <he...@zfabrik.de>]

Hi,

  when running an application that interfaces with an HBase installation
(v. 0.20.6) I noticed that the Hbase shutdown hooks hangs. This does not
happen every time, but frequently. This is the call stack when it hangs:

HCM.shutdownHook:

java.lang.Object.wait(Native Method)
java.lang.Object.wait(Object.java:485)
org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1099)
org.apache.zookeeper.ClientCnxn.close(ClientCnxn.java:1077)
org.apache.zookeeper.ZooKeeper.close(ZooKeeper.java:505)
   - locked org.apache.zookeeper.ZooKeeper@d28057
org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.close(ZooKeeperWrapper.java:681)
org.apache.hadoop.hbase.client.HConnectionManager
$ClientZKWatcher.resetZooKeeper(HConnectionManager.java:233)
   - locked org.apache.hadoop.hbase.client.HConnectionManager
$ClientZKWatcher@1711c2c
org.apache.hadoop.hbase.client.HConnectionManager$ClientZKWatcher.access
$000(HConnectionManager.java:181)
org.apache.hadoop.hbase.client.HConnectionManager.deleteAllConnections(HConnectionManager.java:154)
   - locked java.util.HashMap@1d01254
org.apache.hadoop.hbase.client.HConnectionManager
$1.run(HConnectionManager.java:77)


Is there anything like a time out that would make zoo keeper give up?

Thanks,
  Henning

Re: HBase client shutdown hook hangs

[Stack <st...@duboce.net>]

On Tue, Oct 12, 2010 at 7:06 AM, Henning Blohm <he...@zfabrik.de> wrote:
>  when running an application that interfaces with an HBase installation
> (v. 0.20.6) I noticed that the Hbase shutdown hooks hangs. This does not
> happen every time, but frequently. This is the call stack when it hangs:
>

That looks like the zk client in the hbase client is hanging, waiting
on some notification that is not coming.  Was zk ensemble up at the
time?  Any other earlier exceptions earlier in the logs?

There's an issue to remove shutdown hook in TRUNK.  That'd fix this I
suppose (smile)?

St.Ack