You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-commits@hadoop.apache.org by Apache Wiki <wi...@apache.org> on 2008/07/21 16:35:35 UTC
[Hadoop Wiki] Update of "Hbase/Troubleshooting" by JeanDanielCryans
Dear Wiki user,
You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.
The following page has been changed by JeanDanielCryans:
http://wiki.apache.org/hadoop/Hbase/Troubleshooting
------------------------------------------------------------------------------
=== Resolution ===
* Make sure the HBase root directory does not currently exist or has been initialized by a previous run of HBase. Sure fire solution is to just use Hadoop dfs to delete the HBase root and let HBase create and initialize the directory itself.
+ == Problem: Replay of hlog required, forcing regionserver restart ==
+ * Under a heavy write load, some regions servers will go down with the following exception:
+ {{{
+ WARN org.apache.hadoop.dfs.DFSClient: Exception while reading from blk_xxxxxxxxxxxxxxx of /hbase/some_repository from IP_address:50010: java.io.IOException: Premeture EOF from inputStream
+ then later
+ ERROR org.apache.hadoop.hbase.regionserver.HRegionServer: Error opening scanner (fsOk: true)
+ java.io.IOException: HStoreScanner failed constructionat org.apache.hadoop.hbase.regionserver.StoreFileScanner.<init>(StoreFileScanner.java:69)
+ at org.apache.hadoop.hbase.regionserver.HStoreScanner.<init>(HStoreScanner.java:68)
+ at org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:1896)
+ ...
+ Caused by: java.net.SocketTimeoutException: timed out waiting for rpc response
+ at org.apache.hadoop.ipc.Client.call(Client.java:559)
+ at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
+ at org.apache.hadoop.dfs.$Proxy1.getFileInfo(Unknown Source)
+ }}}
+ === Causes ===
+ * RPC timeouts may happen because of a IO contention which blocks processes during file swapping.
+ === Resolution ===
+ * Either reduce the load or add more memory/machines.
+