You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-commits@hadoop.apache.org by Apache Wiki <wi...@apache.org> on 2010/03/29 19:41:20 UTC
[Hadoop Wiki] Trivial Update of "Hbase/Troubleshooting" by JeanDanielCryans
Dear Wiki user,
You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.
The "Hbase/Troubleshooting" page has been changed by JeanDanielCryans.
http://wiki.apache.org/hadoop/Hbase/Troubleshooting?action=diff&rev1=39&rev2=40
--------------------------------------------------
<<Anchor(9)>>
== 9. Problem: ZooKeeper SessionExpired events ==
+ * Master or Region Servers shutting down with messages like those in the logs:
- * Master or RegionServers reinitialize their ZooKeeper wrappers after receiving SessionExpired events.
- * Master or RegionServer ephemeral nodes disappear while the node is still otherwise functional.
- * Messages those in the logs:
{{{
WARN org.apache.zookeeper.ClientCnxn: Exception
closing session 0x278bd16a96000f to sun.nio.ch.SelectionKeyImpl@355811ec
@@ -186, +184 @@
=== Causes ===
* The JVM is doing a long running garbage collecting which is pausing every threads (aka "stop the world").
* Since the region server's local zookeeper client cannot send heartbeats, the session times out.
+ * By design, we shut down any node that isn't able to contact the Zookeeper ensemble after getting a timeout so that it stops serving data that may already be assigned elsewhere.
=== Resolution ===
* Make sure you give plenty of RAM (in hbase-env.sh), the default of 1GB won't be able to sustain long running imports.
* Make sure you don't swap, the JVM never behaves well under swapping.