You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hbase.apache.org by en...@apache.org on 2013/03/30 01:19:57 UTC
svn commit: r1462679 [8/14] - in /hbase/hbase.apache.org/trunk: ./ book/
case_studies/ community/ configuration/ developer/ getting_started/
ops_mgt/ performance/ rpc/
Modified: hbase/hbase.apache.org/trunk/book/regions.arch.html
URL: http://svn.apache.org/viewvc/hbase/hbase.apache.org/trunk/book/regions.arch.html?rev=1462679&r1=1462678&r2=1462679&view=diff
==============================================================================
--- hbase/hbase.apache.org/trunk/book/regions.arch.html (original)
+++ hbase/hbase.apache.org/trunk/book/regions.arch.html Sat Mar 30 00:19:55 2013
@@ -57,12 +57,12 @@
to the RegionServer.
</p><p>For more information, see <a class="link" href="http://hadoop.apache.org/common/docs/r0.20.205.0/hdfs_design.html#Replica+Placement%3A+The+First+Baby+Steps" target="_top">HDFS Design on Replica Placement</a>
and also Lars George's blog on <a class="link" href="http://www.larsgeorge.com/2010/05/hbase-file-locality-in-hdfs.html" target="_top">HBase and HDFS locality</a>.
- </p></div><div class="section" title="9.7.4. Region Splits"><div class="titlepage"><div><div><h3 class="title"><a name="d2274e5530"></a>9.7.4. Region Splits</h3></div></div></div><p>Splits run unaided on the RegionServer; i.e. the Master does not
+ </p></div><div class="section" title="9.7.4. Region Splits"><div class="titlepage"><div><div><h3 class="title"><a name="d2279e5530"></a>9.7.4. Region Splits</h3></div></div></div><p>Splits run unaided on the RegionServer; i.e. the Master does not
participate. The RegionServer splits a region, offlines the split
region and then adds the daughter regions to META, opens daughters on
the parent's hosting RegionServer and then reports the split to the
Master. See <a class="xref" href="important_configurations.html#disable.splitting" title="2.5.2.7. Managed Splitting">Section 2.5.2.7, “Managed Splitting”</a> for how to manually manage
- splits (and for why you might do this)</p><div class="section" title="9.7.4.1. Custom Split Policies"><div class="titlepage"><div><div><h4 class="title"><a name="d2274e5537"></a>9.7.4.1. Custom Split Policies</h4></div></div></div><p>The default split policy can be overwritten using a custom <a class="link" href="http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/regionserver/RegionSplitPolicy.html" target="_top">RegionSplitPolicy</a> (HBase 0.94+).
+ splits (and for why you might do this)</p><div class="section" title="9.7.4.1. Custom Split Policies"><div class="titlepage"><div><div><h4 class="title"><a name="d2279e5537"></a>9.7.4.1. Custom Split Policies</h4></div></div></div><p>The default split policy can be overwritten using a custom <a class="link" href="http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/regionserver/RegionSplitPolicy.html" target="_top">RegionSplitPolicy</a> (HBase 0.94+).
Typically a custom split policy should extend HBase's default split policy: <a class="link" href="http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/regionserver/ConstantSizeRegionSplitPolicy.html" target="_top">ConstantSizeRegionSplitPolicy</a>.
</p><p>The policy can set globally through the HBaseConfiguration used or on a per table basis:
</p><pre class="programlisting">
@@ -74,7 +74,7 @@ myHtd.setValue(HTableDescriptor.SPLIT_PO
When asked to flush, current memstore is moved to snapshot and is cleared.
HBase continues to serve edits out of new memstore and backing snapshot until flusher reports in that the
flush succeeded. At this point the snapshot is let go.</p></div><div class="section" title="9.7.5.2. StoreFile (HFile)"><div class="titlepage"><div><div><h4 class="title"><a name="hfile"></a>9.7.5.2. StoreFile (HFile)</h4></div></div></div><p>StoreFiles are where your data lives.
- </p><div class="section" title="9.7.5.2.1. HFile Format"><div class="titlepage"><div><div><h5 class="title"><a name="d2274e5568"></a>9.7.5.2.1. HFile Format</h5></div></div></div><p>The <span class="emphasis"><em>hfile</em></span> file format is based on
+ </p><div class="section" title="9.7.5.2.1. HFile Format"><div class="titlepage"><div><div><h5 class="title"><a name="d2279e5568"></a>9.7.5.2.1. HFile Format</h5></div></div></div><p>The <span class="emphasis"><em>hfile</em></span> file format is based on
the SSTable file described in the <a class="link" href="http://research.google.com/archive/bigtable.html" target="_top">BigTable [2006]</a> paper and on
Hadoop's <a class="link" href="http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/io/file/tfile/TFile.html" target="_top">tfile</a>
(The unit test suite and the compression harness were taken directly from tfile).
Modified: hbase/hbase.apache.org/trunk/book/regionserver.arch.html
URL: http://svn.apache.org/viewvc/hbase/hbase.apache.org/trunk/book/regionserver.arch.html?rev=1462679&r1=1462678&r2=1462679&view=diff
==============================================================================
--- hbase/hbase.apache.org/trunk/book/regionserver.arch.html (original)
+++ hbase/hbase.apache.org/trunk/book/regionserver.arch.html Sat Mar 30 00:19:55 2013
@@ -61,22 +61,22 @@
For more general information about the concept of write ahead logs, see the Wikipedia
<a class="link" href="http://en.wikipedia.org/wiki/Write-ahead_logging" target="_top">Write-Ahead Log</a> article.
</p></div><div class="section" title="9.6.5.2. WAL Flushing"><div class="titlepage"><div><div><h4 class="title"><a name="wal_flush"></a>9.6.5.2. WAL Flushing</h4></div></div></div><p>TODO (describe).
- </p></div><div class="section" title="9.6.5.3. WAL Splitting"><div class="titlepage"><div><div><h4 class="title"><a name="wal_splitting"></a>9.6.5.3. WAL Splitting</h4></div></div></div><div class="section" title="9.6.5.3.1. How edits are recovered from a crashed RegionServer"><div class="titlepage"><div><div><h5 class="title"><a name="d2274e5359"></a>9.6.5.3.1. How edits are recovered from a crashed RegionServer</h5></div></div></div><p>When a RegionServer crashes, it will lose its ephemeral lease in
- ZooKeeper...TODO</p></div><div class="section" title="9.6.5.3.2. hbase.hlog.split.skip.errors"><div class="titlepage"><div><div><h5 class="title"><a name="d2274e5364"></a>9.6.5.3.2. <code class="varname">hbase.hlog.split.skip.errors</code></h5></div></div></div><p>When set to <code class="constant">true</code>, any error
+ </p></div><div class="section" title="9.6.5.3. WAL Splitting"><div class="titlepage"><div><div><h4 class="title"><a name="wal_splitting"></a>9.6.5.3. WAL Splitting</h4></div></div></div><div class="section" title="9.6.5.3.1. How edits are recovered from a crashed RegionServer"><div class="titlepage"><div><div><h5 class="title"><a name="d2279e5359"></a>9.6.5.3.1. How edits are recovered from a crashed RegionServer</h5></div></div></div><p>When a RegionServer crashes, it will lose its ephemeral lease in
+ ZooKeeper...TODO</p></div><div class="section" title="9.6.5.3.2. hbase.hlog.split.skip.errors"><div class="titlepage"><div><div><h5 class="title"><a name="d2279e5364"></a>9.6.5.3.2. <code class="varname">hbase.hlog.split.skip.errors</code></h5></div></div></div><p>When set to <code class="constant">true</code>, any error
encountered splitting will be logged, the problematic WAL will be
moved into the <code class="filename">.corrupt</code> directory under the hbase
<code class="varname">rootdir</code>, and processing will continue. If set to
<code class="constant">false</code>, the default, the exception will be propagated and the
- split logged as failed.<sup>[<a name="d2274e5382" href="#ftn.d2274e5382" class="footnote">22</a>]</sup></p></div><div class="section" title="9.6.5.3.3. How EOFExceptions are treated when splitting a crashed RegionServers' WALs"><div class="titlepage"><div><div><h5 class="title"><a name="d2274e5388"></a>9.6.5.3.3. How EOFExceptions are treated when splitting a crashed
+ split logged as failed.<sup>[<a name="d2279e5382" href="#ftn.d2279e5382" class="footnote">22</a>]</sup></p></div><div class="section" title="9.6.5.3.3. How EOFExceptions are treated when splitting a crashed RegionServers' WALs"><div class="titlepage"><div><div><h5 class="title"><a name="d2279e5388"></a>9.6.5.3.3. How EOFExceptions are treated when splitting a crashed
RegionServers' WALs</h5></div></div></div><p>If we get an EOF while splitting logs, we proceed with the split
even when <code class="varname">hbase.hlog.split.skip.errors</code> ==
<code class="constant">false</code>. An EOF while reading the last log in the
set of files to split is near-guaranteed since the RegionServer likely
crashed mid-write of a record. But we'll continue even if we got an
- EOF reading other than the last file in the set.<sup>[<a name="d2274e5399" href="#ftn.d2274e5399" class="footnote">23</a>]</sup></p></div></div></div><div class="footnotes"><br><hr width="100" align="left"><div class="footnote"><p><sup>[<a id="ftn.d2274e5382" href="#d2274e5382" class="para">22</a>] </sup>See <a class="link" href="https://issues.apache.org/jira/browse/HBASE-2958" target="_top">HBASE-2958
+ EOF reading other than the last file in the set.<sup>[<a name="d2279e5399" href="#ftn.d2279e5399" class="footnote">23</a>]</sup></p></div></div></div><div class="footnotes"><br><hr width="100" align="left"><div class="footnote"><p><sup>[<a id="ftn.d2279e5382" href="#d2279e5382" class="para">22</a>] </sup>See <a class="link" href="https://issues.apache.org/jira/browse/HBASE-2958" target="_top">HBASE-2958
When hbase.hlog.split.skip.errors is set to false, we fail the
split but thats it</a>. We need to do more than just fail split
- if this flag is set.</p></div><div class="footnote"><p><sup>[<a id="ftn.d2274e5399" href="#d2274e5399" class="para">23</a>] </sup>For background, see <a class="link" href="https://issues.apache.org/jira/browse/HBASE-2643" target="_top">HBASE-2643
+ if this flag is set.</p></div><div class="footnote"><p><sup>[<a id="ftn.d2279e5399" href="#d2279e5399" class="para">23</a>] </sup>For background, see <a class="link" href="https://issues.apache.org/jira/browse/HBASE-2643" target="_top">HBASE-2643
Figure how to deal with eof splitting logs</a></p></div></div></div><div id="disqus_thread"></div><script type="text/javascript">
var disqus_shortname = 'hbase'; // required: replace example with your forum shortname
var disqus_url = 'http://hbase.apache.org/book';
Modified: hbase/hbase.apache.org/trunk/book/security.html
URL: http://svn.apache.org/viewvc/hbase/hbase.apache.org/trunk/book/security.html?rev=1462679&r1=1462678&r2=1462679&view=diff
==============================================================================
--- hbase/hbase.apache.org/trunk/book/security.html (original)
+++ hbase/hbase.apache.org/trunk/book/security.html Sat Mar 30 00:19:55 2013
@@ -1,6 +1,6 @@
<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
- <title>Chapter 8. Secure Apache HBase (TM)</title><link rel="stylesheet" type="text/css" href="../css/freebsd_docbook.css"><meta name="generator" content="DocBook XSL-NS Stylesheets V1.76.1"><link rel="home" href="book.html" title="The Apache HBase™ Reference Guide"><link rel="up" href="book.html" title="The Apache HBase™ Reference Guide"><link rel="prev" href="mapreduce.specex.html" title="7.4. Speculative Execution"><link rel="next" href="hbase.accesscontrol.configuration.html" title="8.2. Access Control"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="3" align="center">Chapter 8. Secure Apache HBase (TM)</th></tr><tr><td width="20%" align="left"><a accesskey="p" href="mapreduce.specex.html">Prev</a> </td><th width="60%" align="center"> </th><td width="20%" align="right"> <a accesskey="n"
href="hbase.accesscontrol.configuration.html">Next</a></td></tr></table><hr></div><div class="chapter" title="Chapter 8. Secure Apache HBase (TM)"><div class="titlepage"><div><div><h2 class="title"><a name="security"></a>Chapter 8. Secure Apache HBase (TM)</h2></div></div></div><div class="toc"><p><b>Table of Contents</b></p><dl><dt><span class="section"><a href="security.html#hbase.secure.configuration">8.1. Secure Client Access to Apache HBase</a></span></dt><dd><dl><dt><span class="section"><a href="security.html#d2274e4235">8.1.1. Prerequisites</a></span></dt><dt><span class="section"><a href="security.html#d2274e4276">8.1.2. Server-side Configuration for Secure Operation</a></span></dt><dt><span class="section"><a href="security.html#d2274e4288">8.1.3. Client-side Configuration for Secure Operation</a></span></dt><dt><span class="section"><a href="security.html#d2274e4327">8.1.4. Client-side Configuration for Secure Operation - Thrift Gateway</a></sp
an></dt><dt><span class="section"><a href="security.html#d2274e4342">8.1.5. Client-side Configuration for Secure Operation - REST Gateway</a></span></dt></dl></dd><dt><span class="section"><a href="hbase.accesscontrol.configuration.html">8.2. Access Control</a></span></dt><dd><dl><dt><span class="section"><a href="hbase.accesscontrol.configuration.html#d2274e4367">8.2.1. Prerequisites</a></span></dt><dt><span class="section"><a href="hbase.accesscontrol.configuration.html#d2274e4374">8.2.2. Overview</a></span></dt><dt><span class="section"><a href="hbase.accesscontrol.configuration.html#d2274e4531">8.2.3. Server-side Configuration for Access Control</a></span></dt><dt><span class="section"><a href="hbase.accesscontrol.configuration.html#d2274e4543">8.2.4. Shell Enhancements for Access Control</a></span></dt></dl></dd><dt><span class="section"><a href="hbase.secure.bulkload.html">8.3. Secure Bulk Load</a></span></dt></dl></div><div class="section" title="8.1. Secure Clie
nt Access to Apache HBase"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="hbase.secure.configuration"></a>8.1. Secure Client Access to Apache HBase</h2></div></div></div><p>Newer releases of Apache HBase (TM) (>= 0.92) support optional SASL authentication of clients<sup>[<a name="d2274e4226" href="#ftn.d2274e4226" class="footnote">20</a>]</sup>.</p><p>This describes how to set up Apache HBase and clients for connection to secure HBase resources.</p><div class="section" title="8.1.1. Prerequisites"><div class="titlepage"><div><div><h3 class="title"><a name="d2274e4235"></a>8.1.1. Prerequisites</h3></div></div></div><p>
+ <title>Chapter 8. Secure Apache HBase (TM)</title><link rel="stylesheet" type="text/css" href="../css/freebsd_docbook.css"><meta name="generator" content="DocBook XSL-NS Stylesheets V1.76.1"><link rel="home" href="book.html" title="The Apache HBase™ Reference Guide"><link rel="up" href="book.html" title="The Apache HBase™ Reference Guide"><link rel="prev" href="mapreduce.specex.html" title="7.4. Speculative Execution"><link rel="next" href="hbase.accesscontrol.configuration.html" title="8.2. Access Control"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="3" align="center">Chapter 8. Secure Apache HBase (TM)</th></tr><tr><td width="20%" align="left"><a accesskey="p" href="mapreduce.specex.html">Prev</a> </td><th width="60%" align="center"> </th><td width="20%" align="right"> <a accesskey="n"
href="hbase.accesscontrol.configuration.html">Next</a></td></tr></table><hr></div><div class="chapter" title="Chapter 8. Secure Apache HBase (TM)"><div class="titlepage"><div><div><h2 class="title"><a name="security"></a>Chapter 8. Secure Apache HBase (TM)</h2></div></div></div><div class="toc"><p><b>Table of Contents</b></p><dl><dt><span class="section"><a href="security.html#hbase.secure.configuration">8.1. Secure Client Access to Apache HBase</a></span></dt><dd><dl><dt><span class="section"><a href="security.html#d2279e4235">8.1.1. Prerequisites</a></span></dt><dt><span class="section"><a href="security.html#d2279e4276">8.1.2. Server-side Configuration for Secure Operation</a></span></dt><dt><span class="section"><a href="security.html#d2279e4288">8.1.3. Client-side Configuration for Secure Operation</a></span></dt><dt><span class="section"><a href="security.html#d2279e4327">8.1.4. Client-side Configuration for Secure Operation - Thrift Gateway</a></sp
an></dt><dt><span class="section"><a href="security.html#d2279e4342">8.1.5. Client-side Configuration for Secure Operation - REST Gateway</a></span></dt></dl></dd><dt><span class="section"><a href="hbase.accesscontrol.configuration.html">8.2. Access Control</a></span></dt><dd><dl><dt><span class="section"><a href="hbase.accesscontrol.configuration.html#d2279e4367">8.2.1. Prerequisites</a></span></dt><dt><span class="section"><a href="hbase.accesscontrol.configuration.html#d2279e4374">8.2.2. Overview</a></span></dt><dt><span class="section"><a href="hbase.accesscontrol.configuration.html#d2279e4531">8.2.3. Server-side Configuration for Access Control</a></span></dt><dt><span class="section"><a href="hbase.accesscontrol.configuration.html#d2279e4543">8.2.4. Shell Enhancements for Access Control</a></span></dt></dl></dd><dt><span class="section"><a href="hbase.secure.bulkload.html">8.3. Secure Bulk Load</a></span></dt></dl></div><div class="section" title="8.1. Secure Clie
nt Access to Apache HBase"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="hbase.secure.configuration"></a>8.1. Secure Client Access to Apache HBase</h2></div></div></div><p>Newer releases of Apache HBase (TM) (>= 0.92) support optional SASL authentication of clients<sup>[<a name="d2279e4226" href="#ftn.d2279e4226" class="footnote">20</a>]</sup>.</p><p>This describes how to set up Apache HBase and clients for connection to secure HBase resources.</p><div class="section" title="8.1.1. Prerequisites"><div class="titlepage"><div><div><h3 class="title"><a name="d2279e4235"></a>8.1.1. Prerequisites</h3></div></div></div><p>
You need to have a working Kerberos KDC.
</p><p>
A HBase configured for secure client access is expected to be running
@@ -57,7 +57,7 @@
keytabs for Hadoop. Those steps are omitted here. Copy the resulting
keytab files to where the client daemon will execute and make them
readable only to the user account under which the daemon will run.
- </p></div><div class="section" title="8.1.2. Server-side Configuration for Secure Operation"><div class="titlepage"><div><div><h3 class="title"><a name="d2274e4276"></a>8.1.2. Server-side Configuration for Secure Operation</h3></div></div></div><p>
+ </p></div><div class="section" title="8.1.2. Server-side Configuration for Secure Operation"><div class="titlepage"><div><div><h3 class="title"><a name="d2279e4276"></a>8.1.2. Server-side Configuration for Secure Operation</h3></div></div></div><p>
Add the following to the <code class="code">hbase-site.xml</code> file on every server machine in the cluster:
</p><pre class="programlisting">
<property>
@@ -75,7 +75,7 @@
</pre><p>
A full shutdown and restart of HBase service is required when deploying
these configuration changes.
- </p></div><div class="section" title="8.1.3. Client-side Configuration for Secure Operation"><div class="titlepage"><div><div><h3 class="title"><a name="d2274e4288"></a>8.1.3. Client-side Configuration for Secure Operation</h3></div></div></div><p>
+ </p></div><div class="section" title="8.1.3. Client-side Configuration for Secure Operation"><div class="titlepage"><div><div><h3 class="title"><a name="d2279e4288"></a>8.1.3. Client-side Configuration for Secure Operation</h3></div></div></div><p>
Add the following to the <code class="code">hbase-site.xml</code> file on every client:
</p><pre class="programlisting">
<property>
@@ -109,7 +109,7 @@
HTable table = new HTable(conf, tablename);
</pre><p>
Expect a ~10% performance penalty for encrypted communication.
- </p></div><div class="section" title="8.1.4. Client-side Configuration for Secure Operation - Thrift Gateway"><div class="titlepage"><div><div><h3 class="title"><a name="d2274e4327"></a>8.1.4. Client-side Configuration for Secure Operation - Thrift Gateway</h3></div></div></div><p>
+ </p></div><div class="section" title="8.1.4. Client-side Configuration for Secure Operation - Thrift Gateway"><div class="titlepage"><div><div><h3 class="title"><a name="d2279e4327"></a>8.1.4. Client-side Configuration for Secure Operation - Thrift Gateway</h3></div></div></div><p>
Add the following to the <code class="code">hbase-site.xml</code> file for every Thrift gateway:
</p><pre class="programlisting">
<property>
@@ -129,7 +129,7 @@
credential. No authentication will be performed by the Thrift gateway
itself. All client access via the Thrift gateway will use the Thrift
gateway's credential and have its privilege.
- </p></div><div class="section" title="8.1.5. Client-side Configuration for Secure Operation - REST Gateway"><div class="titlepage"><div><div><h3 class="title"><a name="d2274e4342"></a>8.1.5. Client-side Configuration for Secure Operation - REST Gateway</h3></div></div></div><p>
+ </p></div><div class="section" title="8.1.5. Client-side Configuration for Secure Operation - REST Gateway"><div class="titlepage"><div><div><h3 class="title"><a name="d2279e4342"></a>8.1.5. Client-side Configuration for Secure Operation - REST Gateway</h3></div></div></div><p>
Add the following to the <code class="code">hbase-site.xml</code> file for every REST gateway:
</p><pre class="programlisting">
<property>
@@ -153,7 +153,7 @@
It should be possible for clients to authenticate with the HBase
cluster through the REST gateway in a pass-through manner via SPEGNO
HTTP authentication. This is future work.
- </p></div></div><div class="footnotes"><br><hr width="100" align="left"><div class="footnote"><p><sup>[<a id="ftn.d2274e4226" href="#d2274e4226" class="para">20</a>] </sup>See
+ </p></div></div><div class="footnotes"><br><hr width="100" align="left"><div class="footnote"><p><sup>[<a id="ftn.d2279e4226" href="#d2279e4226" class="para">20</a>] </sup>See
also Matteo Bertozzi's article on <a class="link" href="http://www.cloudera.com/blog/2012/09/understanding-user-authentication-and-authorization-in-apache-hbase/" target="_top">Understanding User Authentication and Authorization in Apache HBase</a>.</p></div></div></div><div id="disqus_thread"></div><script type="text/javascript">
var disqus_shortname = 'hbase'; // required: replace example with your forum shortname
var disqus_url = 'http://hbase.apache.org/book';
Modified: hbase/hbase.apache.org/trunk/book/shell.html
URL: http://svn.apache.org/viewvc/hbase/hbase.apache.org/trunk/book/shell.html?rev=1462679&r1=1462678&r2=1462679&view=diff
==============================================================================
--- hbase/hbase.apache.org/trunk/book/shell.html (original)
+++ hbase/hbase.apache.org/trunk/book/shell.html Sat Mar 30 00:19:55 2013
@@ -1,6 +1,6 @@
<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
- <title>Chapter 4. The Apache HBase Shell</title><link rel="stylesheet" type="text/css" href="../css/freebsd_docbook.css"><meta name="generator" content="DocBook XSL-NS Stylesheets V1.76.1"><link rel="home" href="book.html" title="The Apache HBase™ Reference Guide"><link rel="up" href="book.html" title="The Apache HBase™ Reference Guide"><link rel="prev" href="upgrade0.90.html" title="3.4. Upgrading to HBase 0.90.x from 0.20.x or 0.89.x"><link rel="next" href="shell_tricks.html" title="4.2. Shell Tricks"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="3" align="center">Chapter 4. The Apache HBase Shell</th></tr><tr><td width="20%" align="left"><a accesskey="p" href="upgrade0.90.html">Prev</a> </td><th width="60%" align="center"> </th><td width="20%" align="right"> <a accesskey="n" href="shell
_tricks.html">Next</a></td></tr></table><hr></div><div class="chapter" title="Chapter 4. The Apache HBase Shell"><div class="titlepage"><div><div><h2 class="title"><a name="shell"></a>Chapter 4. The Apache HBase Shell</h2></div></div></div><div class="toc"><p><b>Table of Contents</b></p><dl><dt><span class="section"><a href="shell.html#scripting">4.1. Scripting</a></span></dt><dt><span class="section"><a href="shell_tricks.html">4.2. Shell Tricks</a></span></dt><dd><dl><dt><span class="section"><a href="shell_tricks.html#d2274e2913">4.2.1. <code class="filename">irbrc</code></a></span></dt><dt><span class="section"><a href="shell_tricks.html#d2274e2931">4.2.2. LOG data to timestamp</a></span></dt><dt><span class="section"><a href="shell_tricks.html#d2274e2949">4.2.3. Debug</a></span></dt><dt><span class="section"><a href="shell_tricks.html#d2274e2971">4.2.4. Commands</a></span></dt></dl></dd></dl></div><p>
+ <title>Chapter 4. The Apache HBase Shell</title><link rel="stylesheet" type="text/css" href="../css/freebsd_docbook.css"><meta name="generator" content="DocBook XSL-NS Stylesheets V1.76.1"><link rel="home" href="book.html" title="The Apache HBase™ Reference Guide"><link rel="up" href="book.html" title="The Apache HBase™ Reference Guide"><link rel="prev" href="upgrade0.90.html" title="3.4. Upgrading to HBase 0.90.x from 0.20.x or 0.89.x"><link rel="next" href="shell_tricks.html" title="4.2. Shell Tricks"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="3" align="center">Chapter 4. The Apache HBase Shell</th></tr><tr><td width="20%" align="left"><a accesskey="p" href="upgrade0.90.html">Prev</a> </td><th width="60%" align="center"> </th><td width="20%" align="right"> <a accesskey="n" href="shell
_tricks.html">Next</a></td></tr></table><hr></div><div class="chapter" title="Chapter 4. The Apache HBase Shell"><div class="titlepage"><div><div><h2 class="title"><a name="shell"></a>Chapter 4. The Apache HBase Shell</h2></div></div></div><div class="toc"><p><b>Table of Contents</b></p><dl><dt><span class="section"><a href="shell.html#scripting">4.1. Scripting</a></span></dt><dt><span class="section"><a href="shell_tricks.html">4.2. Shell Tricks</a></span></dt><dd><dl><dt><span class="section"><a href="shell_tricks.html#d2279e2913">4.2.1. <code class="filename">irbrc</code></a></span></dt><dt><span class="section"><a href="shell_tricks.html#d2279e2931">4.2.2. LOG data to timestamp</a></span></dt><dt><span class="section"><a href="shell_tricks.html#d2279e2949">4.2.3. Debug</a></span></dt><dt><span class="section"><a href="shell_tricks.html#d2279e2971">4.2.4. Commands</a></span></dt></dl></dd></dl></div><p>
The Apache HBase (TM) Shell is <a class="link" href="http://jruby.org" target="_top">(J)Ruby</a>'s
IRB with some HBase particular commands added. Anything you can do in
IRB, you should be able to do in the HBase Shell.</p><p>To run the HBase shell,
Modified: hbase/hbase.apache.org/trunk/book/shell_tricks.html
URL: http://svn.apache.org/viewvc/hbase/hbase.apache.org/trunk/book/shell_tricks.html?rev=1462679&r1=1462678&r2=1462679&view=diff
==============================================================================
--- hbase/hbase.apache.org/trunk/book/shell_tricks.html (original)
+++ hbase/hbase.apache.org/trunk/book/shell_tricks.html Sat Mar 30 00:19:55 2013
@@ -1,6 +1,6 @@
<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
- <title>4.2. Shell Tricks</title><link rel="stylesheet" type="text/css" href="../css/freebsd_docbook.css"><meta name="generator" content="DocBook XSL-NS Stylesheets V1.76.1"><link rel="home" href="book.html" title="The Apache HBase™ Reference Guide"><link rel="up" href="shell.html" title="Chapter 4. The Apache HBase Shell"><link rel="prev" href="shell.html" title="Chapter 4. The Apache HBase Shell"><link rel="next" href="datamodel.html" title="Chapter 5. Data Model"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="3" align="center">4.2. Shell Tricks</th></tr><tr><td width="20%" align="left"><a accesskey="p" href="shell.html">Prev</a> </td><th width="60%" align="center">Chapter 4. The Apache HBase Shell</th><td width="20%" align="right"> <a accesskey="n" href="datamodel.html">Next</a>
</td></tr></table><hr></div><div class="section" title="4.2. Shell Tricks"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="shell_tricks"></a>4.2. Shell Tricks</h2></div></div></div><div class="section" title="4.2.1. irbrc"><div class="titlepage"><div><div><h3 class="title"><a name="d2274e2913"></a>4.2.1. <code class="filename">irbrc</code></h3></div></div></div><p>Create an <code class="filename">.irbrc</code> file for yourself in your
+ <title>4.2. Shell Tricks</title><link rel="stylesheet" type="text/css" href="../css/freebsd_docbook.css"><meta name="generator" content="DocBook XSL-NS Stylesheets V1.76.1"><link rel="home" href="book.html" title="The Apache HBase™ Reference Guide"><link rel="up" href="shell.html" title="Chapter 4. The Apache HBase Shell"><link rel="prev" href="shell.html" title="Chapter 4. The Apache HBase Shell"><link rel="next" href="datamodel.html" title="Chapter 5. Data Model"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="3" align="center">4.2. Shell Tricks</th></tr><tr><td width="20%" align="left"><a accesskey="p" href="shell.html">Prev</a> </td><th width="60%" align="center">Chapter 4. The Apache HBase Shell</th><td width="20%" align="right"> <a accesskey="n" href="datamodel.html">Next</a>
</td></tr></table><hr></div><div class="section" title="4.2. Shell Tricks"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="shell_tricks"></a>4.2. Shell Tricks</h2></div></div></div><div class="section" title="4.2.1. irbrc"><div class="titlepage"><div><div><h3 class="title"><a name="d2279e2913"></a>4.2.1. <code class="filename">irbrc</code></h3></div></div></div><p>Create an <code class="filename">.irbrc</code> file for yourself in your
home directory. Add customizations. A useful one is
command history so commands are save across Shell invocations:
</p><pre class="programlisting">
@@ -11,7 +11,7 @@
See the <span class="application">ruby</span> documentation of
<code class="filename">.irbrc</code> to learn about other possible
confiurations.
- </p></div><div class="section" title="4.2.2. LOG data to timestamp"><div class="titlepage"><div><div><h3 class="title"><a name="d2274e2931"></a>4.2.2. LOG data to timestamp</h3></div></div></div><p>
+ </p></div><div class="section" title="4.2.2. LOG data to timestamp"><div class="titlepage"><div><div><h3 class="title"><a name="d2279e2931"></a>4.2.2. LOG data to timestamp</h3></div></div></div><p>
To convert the date '08/08/16 20:56:29' from an hbase log into a timestamp, do:
</p><pre class="programlisting">
hbase(main):021:0> import java.text.SimpleDateFormat
@@ -25,14 +25,14 @@
</p><p>
To output in a format that is exactly like that of the HBase log format will take a little messing with
<a class="link" href="http://download.oracle.com/javase/6/docs/api/java/text/SimpleDateFormat.html" target="_top">SimpleDateFormat</a>.
- </p></div><div class="section" title="4.2.3. Debug"><div class="titlepage"><div><div><h3 class="title"><a name="d2274e2949"></a>4.2.3. Debug</h3></div></div></div><div class="section" title="4.2.3.1. Shell debug switch"><div class="titlepage"><div><div><h4 class="title"><a name="d2274e2952"></a>4.2.3.1. Shell debug switch</h4></div></div></div><p>You can set a debug switch in the shell to see more output
+ </p></div><div class="section" title="4.2.3. Debug"><div class="titlepage"><div><div><h3 class="title"><a name="d2279e2949"></a>4.2.3. Debug</h3></div></div></div><div class="section" title="4.2.3.1. Shell debug switch"><div class="titlepage"><div><div><h4 class="title"><a name="d2279e2952"></a>4.2.3.1. Shell debug switch</h4></div></div></div><p>You can set a debug switch in the shell to see more output
-- e.g. more of the stack trace on exception --
when you run a command:
</p><pre class="programlisting">hbase> debug <RETURN></pre><p>
- </p></div><div class="section" title="4.2.3.2. DEBUG log level"><div class="titlepage"><div><div><h4 class="title"><a name="d2274e2960"></a>4.2.3.2. DEBUG log level</h4></div></div></div><p>To enable DEBUG level logging in the shell,
+ </p></div><div class="section" title="4.2.3.2. DEBUG log level"><div class="titlepage"><div><div><h4 class="title"><a name="d2279e2960"></a>4.2.3.2. DEBUG log level</h4></div></div></div><p>To enable DEBUG level logging in the shell,
launch it with the <span class="command"><strong>-d</strong></span> option.
</p><pre class="programlisting">$ ./bin/hbase shell -d</pre><p>
- </p></div></div><div class="section" title="4.2.4. Commands"><div class="titlepage"><div><div><h3 class="title"><a name="d2274e2971"></a>4.2.4. Commands</h3></div></div></div><div class="section" title="4.2.4.1. count"><div class="titlepage"><div><div><h4 class="title"><a name="d2274e2974"></a>4.2.4.1. count</h4></div></div></div><p>Count command returns the number of rows in a table.
+ </p></div></div><div class="section" title="4.2.4. Commands"><div class="titlepage"><div><div><h3 class="title"><a name="d2279e2971"></a>4.2.4. Commands</h3></div></div></div><div class="section" title="4.2.4.1. count"><div class="titlepage"><div><div><h4 class="title"><a name="d2279e2974"></a>4.2.4.1. count</h4></div></div></div><p>Count command returns the number of rows in a table.
It's quite fast when configured with the right CACHE
</p><pre class="programlisting">hbase> count '<tablename>', CACHE => 1000</pre><p>
The above count fetches 1000 rows at a time. Set CACHE lower if your rows are big.
Modified: hbase/hbase.apache.org/trunk/book/snappy.compression.html
URL: http://svn.apache.org/viewvc/hbase/hbase.apache.org/trunk/book/snappy.compression.html?rev=1462679&r1=1462678&r2=1462679&view=diff
==============================================================================
--- hbase/hbase.apache.org/trunk/book/snappy.compression.html (original)
+++ hbase/hbase.apache.org/trunk/book/snappy.compression.html Sat Mar 30 00:19:55 2013
@@ -7,7 +7,7 @@
</h2></div></div></div><p>
If snappy is installed, HBase can make use of it (courtesy of
<a class="link" href="http://code.google.com/p/hadoop-snappy/" target="_top">hadoop-snappy</a>
- <sup>[<a name="d2274e11916" href="#ftn.d2274e11916" class="footnote">32</a>]</sup>).
+ <sup>[<a name="d2279e11930" href="#ftn.d2279e11930" class="footnote">32</a>]</sup>).
</p><div class="orderedlist"><ol class="orderedlist" type="1"><li class="listitem"><p>
Build and install <a class="link" href="http://code.google.com/p/snappy/" target="_top">snappy</a> on all nodes
@@ -45,7 +45,7 @@ hbase> describe 't1'</pre><p>
libsnappy.so.1
libsnappy.so.1.1.2
</pre><p>
- </p></div><div class="footnotes"><br><hr width="100" align="left"><div class="footnote"><p><sup>[<a id="ftn.d2274e11916" href="#d2274e11916" class="para">32</a>] </sup>See <a class="link" href="http://search-hadoop.com/m/Ds8d51c263B1/%2522Hadoop-Snappy+in+synch+with+Hadoop+trunk%2522&subj=Hadoop+Snappy+in+synch+with+Hadoop+trunk" target="_top">Alejandro's note</a> up on the list on difference between Snappy in Hadoop
+ </p></div><div class="footnotes"><br><hr width="100" align="left"><div class="footnote"><p><sup>[<a id="ftn.d2279e11930" href="#d2279e11930" class="para">32</a>] </sup>See <a class="link" href="http://search-hadoop.com/m/Ds8d51c263B1/%2522Hadoop-Snappy+in+synch+with+Hadoop+trunk%2522&subj=Hadoop+Snappy+in+synch+with+Hadoop+trunk" target="_top">Alejandro's note</a> up on the list on difference between Snappy in Hadoop
and Snappy in HBase</p></div></div></div><div id="disqus_thread"></div><script type="text/javascript">
var disqus_shortname = 'hbase'; // required: replace example with your forum shortname
var disqus_url = 'http://hbase.apache.org/book';
Modified: hbase/hbase.apache.org/trunk/book/standalone_dist.html
URL: http://svn.apache.org/viewvc/hbase/hbase.apache.org/trunk/book/standalone_dist.html?rev=1462679&r1=1462678&r2=1462679&view=diff
==============================================================================
--- hbase/hbase.apache.org/trunk/book/standalone_dist.html (original)
+++ hbase/hbase.apache.org/trunk/book/standalone_dist.html Sat Mar 30 00:19:55 2013
@@ -18,7 +18,7 @@
daemons run on a single node -- a.k.a
<span class="emphasis"><em>pseudo-distributed</em></span>-- and
<span class="emphasis"><em>fully-distributed</em></span> where the daemons are spread
- across all nodes in the cluster <sup>[<a name="d2274e717" href="#ftn.d2274e717" class="footnote">9</a>]</sup>.</p><p>Distributed modes require an instance of the <span class="emphasis"><em>Hadoop
+ across all nodes in the cluster <sup>[<a name="d2279e717" href="#ftn.d2279e717" class="footnote">9</a>]</sup>.</p><p>Distributed modes require an instance of the <span class="emphasis"><em>Hadoop
Distributed File System</em></span> (HDFS). See the Hadoop <a class="link" href="http://hadoop.apache.org/common/docs/r1.1.1/api/overview-summary.html#overview_description" target="_top">
requirements and instructions</a> for how to set up a HDFS. Before
proceeding, ensure you have an appropriate, working HDFS.</p><p>Below we describe the different distributed setups. Starting,
@@ -37,7 +37,7 @@
Note that the <code class="varname">hbase.rootdir</code> property points to the
local HDFS instance.
</p><p>Now skip to <a class="xref" href="standalone_dist.html#confirm" title="2.2.3. Running and Confirming Your Installation">Section 2.2.3, “Running and Confirming Your Installation”</a> for how to start and verify your
- pseudo-distributed install. <sup>[<a name="d2274e765" href="#ftn.d2274e765" class="footnote">10</a>]</sup></p><div class="note" title="Note" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Note</h3><p>Let HBase create the <code class="varname">hbase.rootdir</code>
+ pseudo-distributed install. <sup>[<a name="d2279e765" href="#ftn.d2279e765" class="footnote">10</a>]</sup></p><div class="note" title="Note" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Note</h3><p>Let HBase create the <code class="varname">hbase.rootdir</code>
directory. If you don't, you'll get warning saying HBase needs a
migration run because the directory is missing files expected by
HBase (it'll create them if you let it).</p></div><div class="section" title="2.2.2.1.1. Pseudo-distributed Configuration File"><div class="titlepage"><div><div><h5 class="title"><a name="pseudo.config"></a>2.2.2.1.1. Pseudo-distributed Configuration File</h5></div></div></div><p>Below is a sample pseudo-distributed file for the node <code class="varname">h-24-30.example.com</code>.
@@ -158,8 +158,8 @@ stopping hbase...............</pre><p> S
complete. It can take longer if your cluster is comprised of many
machines. If you are running a distributed operation, be sure to wait
until HBase has shut down completely before stopping the Hadoop
- daemons.</p></div><div class="footnotes"><br><hr width="100" align="left"><div class="footnote"><p><sup>[<a id="ftn.d2274e717" href="#d2274e717" class="para">9</a>] </sup>The pseudo-distributed vs fully-distributed nomenclature
- comes from Hadoop.</p></div><div class="footnote"><p><sup>[<a id="ftn.d2274e765" href="#d2274e765" class="para">10</a>] </sup>See <a class="xref" href="standalone_dist.html#pseudo.extras" title="2.2.2.1.2. Pseudo-distributed Extras">Section 2.2.2.1.2, “Pseudo-distributed Extras”</a> for notes on how to start extra Masters and
+ daemons.</p></div><div class="footnotes"><br><hr width="100" align="left"><div class="footnote"><p><sup>[<a id="ftn.d2279e717" href="#d2279e717" class="para">9</a>] </sup>The pseudo-distributed vs fully-distributed nomenclature
+ comes from Hadoop.</p></div><div class="footnote"><p><sup>[<a id="ftn.d2279e765" href="#d2279e765" class="para">10</a>] </sup>See <a class="xref" href="standalone_dist.html#pseudo.extras" title="2.2.2.1.2. Pseudo-distributed Extras">Section 2.2.2.1.2, “Pseudo-distributed Extras”</a> for notes on how to start extra Masters and
RegionServers when running pseudo-distributed.</p></div></div></div><div id="disqus_thread"></div><script type="text/javascript">
var disqus_shortname = 'hbase'; // required: replace example with your forum shortname
var disqus_url = 'http://hbase.apache.org/book';
Modified: hbase/hbase.apache.org/trunk/book/trouble.client.html
URL: http://svn.apache.org/viewvc/hbase/hbase.apache.org/trunk/book/trouble.client.html?rev=1462679&r1=1462678&r2=1462679&view=diff
==============================================================================
--- hbase/hbase.apache.org/trunk/book/trouble.client.html (original)
+++ hbase/hbase.apache.org/trunk/book/trouble.client.html Sat Mar 30 00:19:55 2013
@@ -5,7 +5,7 @@
For example, if <code class="code">Scan.setCaching</code> is set to 500, then there will be an RPC call to fetch the next batch of rows every 500 <code class="code">.next()</code> calls on the ResultScanner
because data is being transferred in blocks of 500 rows to the client. Reducing the setCaching value may be an option, but setting this value too low makes for inefficient
processing on numbers of rows.
- </p><p>See <a class="xref" href="perf.reading.html#perf.hbase.client.caching" title="11.8.1. Scan Caching">Section 11.8.1, “Scan Caching”</a>.
+ </p><p>See <a class="xref" href="perf.reading.html#perf.hbase.client.caching" title="11.9.1. Scan Caching">Section 11.9.1, “Scan Caching”</a>.
</p></div><div class="section" title="12.5.2. LeaseException when calling Scanner.next"><div class="titlepage"><div><div><h3 class="title"><a name="trouble.client.lease.exception"></a>12.5.2. <code class="classname">LeaseException</code> when calling <code class="classname">Scanner.next</code></h3></div></div></div><p>
In some situations clients that fetch data from a RegionServer get a LeaseException instead of the usual
<a class="xref" href="trouble.client.html#trouble.client.scantimeout" title="12.5.1. ScannerTimeoutException or UnknownScannerException">Section 12.5.1, “ScannerTimeoutException or UnknownScannerException”</a>. Usually the source of the exception is
@@ -17,7 +17,7 @@ Harsh J investigated the issue as part o
</p></div><div class="section" title="12.5.3. Shell or client application throws lots of scary exceptions during normal operation"><div class="titlepage"><div><div><h3 class="title"><a name="trouble.client.scarylogs"></a>12.5.3. Shell or client application throws lots of scary exceptions during normal operation</h3></div></div></div><p>Since 0.20.0 the default log level for <code class="code">org.apache.hadoop.hbase.*</code>is DEBUG. </p><p>
On your clients, edit <code class="filename">$HBASE_HOME/conf/log4j.properties</code> and change this: <code class="code">log4j.logger.org.apache.hadoop.hbase=DEBUG</code> to this: <code class="code">log4j.logger.org.apache.hadoop.hbase=INFO</code>, or even <code class="code">log4j.logger.org.apache.hadoop.hbase=WARN</code>.
</p></div><div class="section" title="12.5.4. Long Client Pauses With Compression"><div class="titlepage"><div><div><h3 class="title"><a name="trouble.client.longpauseswithcompression"></a>12.5.4. Long Client Pauses With Compression</h3></div></div></div><p>This is a fairly frequent question on the Apache HBase dist-list. The scenario is that a client is typically inserting a lot of data into a
- relatively un-optimized HBase cluster. Compression can exacerbate the pauses, although it is not the source of the problem.</p><p>See <a class="xref" href="perf.writing.html#precreate.regions" title="11.7.2. Table Creation: Pre-Creating Regions">Section 11.7.2, “
+ relatively un-optimized HBase cluster. Compression can exacerbate the pauses, although it is not the source of the problem.</p><p>See <a class="xref" href="perf.writing.html#precreate.regions" title="11.8.2. Table Creation: Pre-Creating Regions">Section 11.8.2, “
Table Creation: Pre-Creating Regions
”</a> on the pattern for pre-creating regions and confirm that the table isn't starting with a single region.</p><p>See <a class="xref" href="perf.configurations.html" title="11.4. HBase Configurations">Section 11.4, “HBase Configurations”</a> for cluster configuration, particularly <code class="code">hbase.hstore.blockingStoreFiles</code>, <code class="code">hbase.hregion.memstore.block.multiplier</code>,
<code class="code">MAX_FILESIZE</code> (region size), and <code class="code">MEMSTORE_FLUSHSIZE.</code> </p><p>A slightly longer explanation of why pauses can happen is as follows: Puts are sometimes blocked on the MemStores which are blocked by the flusher thread which is blocked because there are
Modified: hbase/hbase.apache.org/trunk/book/trouble.html
URL: http://svn.apache.org/viewvc/hbase/hbase.apache.org/trunk/book/trouble.html?rev=1462679&r1=1462678&r2=1462679&view=diff
==============================================================================
--- hbase/hbase.apache.org/trunk/book/trouble.html (original)
+++ hbase/hbase.apache.org/trunk/book/trouble.html Sat Mar 30 00:19:55 2013
@@ -1,6 +1,6 @@
<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
- <title>Chapter 12. Troubleshooting and Debugging Apache HBase (TM)</title><link rel="stylesheet" type="text/css" href="../css/freebsd_docbook.css"><meta name="generator" content="DocBook XSL-NS Stylesheets V1.76.1"><link rel="home" href="book.html" title="The Apache HBase™ Reference Guide"><link rel="up" href="book.html" title="The Apache HBase™ Reference Guide"><link rel="prev" href="perf.casestudy.html" title="11.12. Case Studies"><link rel="next" href="trouble.log.html" title="12.2. Logs"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="3" align="center">Chapter 12. Troubleshooting and Debugging Apache HBase (TM)</th></tr><tr><td width="20%" align="left"><a accesskey="p" href="perf.casestudy.html">Prev</a> </td><th width="60%" align="center"> </th><td width="20%" align="right"> <a accesske
y="n" href="trouble.log.html">Next</a></td></tr></table><hr></div><div class="chapter" title="Chapter 12. Troubleshooting and Debugging Apache HBase (TM)"><div class="titlepage"><div><div><h2 class="title"><a name="trouble"></a>Chapter 12. Troubleshooting and Debugging Apache HBase (TM)</h2></div></div></div><div class="toc"><p><b>Table of Contents</b></p><dl><dt><span class="section"><a href="trouble.html#trouble.general">12.1. General Guidelines</a></span></dt><dt><span class="section"><a href="trouble.log.html">12.2. Logs</a></span></dt><dd><dl><dt><span class="section"><a href="trouble.log.html#trouble.log.locations">12.2.1. Log Locations</a></span></dt><dt><span class="section"><a href="trouble.log.html#trouble.log.levels">12.2.2. Log Levels</a></span></dt><dt><span class="section"><a href="trouble.log.html#trouble.log.gc">12.2.3. JVM Garbage Collection Logs</a></span></dt></dl></dd><dt><span class="section"><a href="trouble.resources.html">12.3. Res
ources</a></span></dt><dd><dl><dt><span class="section"><a href="trouble.resources.html#trouble.resources.searchhadoop">12.3.1. search-hadoop.com</a></span></dt><dt><span class="section"><a href="trouble.resources.html#trouble.resources.lists">12.3.2. Mailing Lists</a></span></dt><dt><span class="section"><a href="trouble.resources.html#trouble.resources.irc">12.3.3. IRC</a></span></dt><dt><span class="section"><a href="trouble.resources.html#trouble.resources.jira">12.3.4. JIRA</a></span></dt></dl></dd><dt><span class="section"><a href="trouble.tools.html">12.4. Tools</a></span></dt><dd><dl><dt><span class="section"><a href="trouble.tools.html#trouble.tools.builtin">12.4.1. Builtin Tools</a></span></dt><dt><span class="section"><a href="trouble.tools.html#trouble.tools.external">12.4.2. External Tools</a></span></dt></dl></dd><dt><span class="section"><a href="trouble.client.html">12.5. Client</a></span></dt><dd><dl><dt><span class="section"><a href="trouble.client.html#tro
uble.client.scantimeout">12.5.1. ScannerTimeoutException or UnknownScannerException</a></span></dt><dt><span class="section"><a href="trouble.client.html#trouble.client.lease.exception">12.5.2. <code class="classname">LeaseException</code> when calling <code class="classname">Scanner.next</code></a></span></dt><dt><span class="section"><a href="trouble.client.html#trouble.client.scarylogs">12.5.3. Shell or client application throws lots of scary exceptions during normal operation</a></span></dt><dt><span class="section"><a href="trouble.client.html#trouble.client.longpauseswithcompression">12.5.4. Long Client Pauses With Compression</a></span></dt><dt><span class="section"><a href="trouble.client.html#trouble.client.zookeeper">12.5.5. ZooKeeper Client Connection Errors</a></span></dt><dt><span class="section"><a href="trouble.client.html#trouble.client.oome.directmemory.leak">12.5.6. Client running out of memory though heap size seems to be stable (but the off-heap/direct he
ap keeps growing)</a></span></dt><dt><span class="section"><a href="trouble.client.html#trouble.client.slowdown.admin">12.5.7. Client Slowdown When Calling Admin Methods (flush, compact, etc.)</a></span></dt><dt><span class="section"><a href="trouble.client.html#trouble.client.security.rpc">12.5.8. Secure Client Cannot Connect ([Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)])</a></span></dt></dl></dd><dt><span class="section"><a href="trouble.mapreduce.html">12.6. MapReduce</a></span></dt><dd><dl><dt><span class="section"><a href="trouble.mapreduce.html#trouble.mapreduce.local">12.6.1. You Think You're On The Cluster, But You're Actually Local</a></span></dt></dl></dd><dt><span class="section"><a href="trouble.namenode.html">12.7. NameNode</a></span></dt><dd><dl><dt><span class="section"><a href="trouble.namenode.html#trouble.namenode.disk">12.7.1. HDFS Utilization of Tables and Regions</a></span></dt><dt><span class
="section"><a href="trouble.namenode.html#trouble.namenode.hbase.objects">12.7.2. Browsing HDFS for HBase Objects</a></span></dt></dl></dd><dt><span class="section"><a href="trouble.network.html">12.8. Network</a></span></dt><dd><dl><dt><span class="section"><a href="trouble.network.html#trouble.network.spikes">12.8.1. Network Spikes</a></span></dt><dt><span class="section"><a href="trouble.network.html#trouble.network.loopback">12.8.2. Loopback IP</a></span></dt><dt><span class="section"><a href="trouble.network.html#trouble.network.ints">12.8.3. Network Interfaces</a></span></dt></dl></dd><dt><span class="section"><a href="trouble.rs.html">12.9. RegionServer</a></span></dt><dd><dl><dt><span class="section"><a href="trouble.rs.html#trouble.rs.startup">12.9.1. Startup Errors</a></span></dt><dt><span class="section"><a href="trouble.rs.html#trouble.rs.runtime">12.9.2. Runtime Errors</a></span></dt><dt><span class="section"><a href="trouble.rs.html#trouble.rs.shutdown">12.9.3.
Shutdown Errors</a></span></dt></dl></dd><dt><span class="section"><a href="trouble.master.html">12.10. Master</a></span></dt><dd><dl><dt><span class="section"><a href="trouble.master.html#trouble.master.startup">12.10.1. Startup Errors</a></span></dt><dt><span class="section"><a href="trouble.master.html#trouble.master.shutdown">12.10.2. Shutdown Errors</a></span></dt></dl></dd><dt><span class="section"><a href="trouble.zookeeper.html">12.11. ZooKeeper</a></span></dt><dd><dl><dt><span class="section"><a href="trouble.zookeeper.html#trouble.zookeeper.startup">12.11.1. Startup Errors</a></span></dt><dt><span class="section"><a href="trouble.zookeeper.html#trouble.zookeeper.general">12.11.2. ZooKeeper, The Cluster Canary</a></span></dt></dl></dd><dt><span class="section"><a href="trouble.ec2.html">12.12. Amazon EC2</a></span></dt><dd><dl><dt><span class="section"><a href="trouble.ec2.html#trouble.ec2.zookeeper">12.12.1. ZooKeeper does not seem to work on Amazon EC2</a></span>
</dt><dt><span class="section"><a href="trouble.ec2.html#trouble.ec2.instability">12.12.2. Instability on Amazon EC2</a></span></dt><dt><span class="section"><a href="trouble.ec2.html#trouble.ec2.connection">12.12.3. Remote Java Connection into EC2 Cluster Not Working</a></span></dt></dl></dd><dt><span class="section"><a href="trouble.versions.html">12.13. HBase and Hadoop version issues</a></span></dt><dd><dl><dt><span class="section"><a href="trouble.versions.html#trouble.versions.205">12.13.1. <code class="code">NoClassDefFoundError</code> when trying to run 0.90.x on hadoop-0.20.205.x (or hadoop-1.0.x)</a></span></dt><dt><span class="section"><a href="trouble.versions.html#trouble.versions.205">12.13.2. ...cannot communicate with client version...</a></span></dt></dl></dd><dt><span class="section"><a href="trouble.casestudy.html">12.14. Case Studies</a></span></dt></dl></div><div class="section" title="12.1. General Guidelines"><div class="titlepage"><div><div><h2 c
lass="title" style="clear: both"><a name="trouble.general"></a>12.1. General Guidelines</h2></div></div></div><p>
+ <title>Chapter 12. Troubleshooting and Debugging Apache HBase (TM)</title><link rel="stylesheet" type="text/css" href="../css/freebsd_docbook.css"><meta name="generator" content="DocBook XSL-NS Stylesheets V1.76.1"><link rel="home" href="book.html" title="The Apache HBase™ Reference Guide"><link rel="up" href="book.html" title="The Apache HBase™ Reference Guide"><link rel="prev" href="perf.casestudy.html" title="11.13. Case Studies"><link rel="next" href="trouble.log.html" title="12.2. Logs"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="3" align="center">Chapter 12. Troubleshooting and Debugging Apache HBase (TM)</th></tr><tr><td width="20%" align="left"><a accesskey="p" href="perf.casestudy.html">Prev</a> </td><th width="60%" align="center"> </th><td width="20%" align="right"> <a accesske
y="n" href="trouble.log.html">Next</a></td></tr></table><hr></div><div class="chapter" title="Chapter 12. Troubleshooting and Debugging Apache HBase (TM)"><div class="titlepage"><div><div><h2 class="title"><a name="trouble"></a>Chapter 12. Troubleshooting and Debugging Apache HBase (TM)</h2></div></div></div><div class="toc"><p><b>Table of Contents</b></p><dl><dt><span class="section"><a href="trouble.html#trouble.general">12.1. General Guidelines</a></span></dt><dt><span class="section"><a href="trouble.log.html">12.2. Logs</a></span></dt><dd><dl><dt><span class="section"><a href="trouble.log.html#trouble.log.locations">12.2.1. Log Locations</a></span></dt><dt><span class="section"><a href="trouble.log.html#trouble.log.levels">12.2.2. Log Levels</a></span></dt><dt><span class="section"><a href="trouble.log.html#trouble.log.gc">12.2.3. JVM Garbage Collection Logs</a></span></dt></dl></dd><dt><span class="section"><a href="trouble.resources.html">12.3. Res
ources</a></span></dt><dd><dl><dt><span class="section"><a href="trouble.resources.html#trouble.resources.searchhadoop">12.3.1. search-hadoop.com</a></span></dt><dt><span class="section"><a href="trouble.resources.html#trouble.resources.lists">12.3.2. Mailing Lists</a></span></dt><dt><span class="section"><a href="trouble.resources.html#trouble.resources.irc">12.3.3. IRC</a></span></dt><dt><span class="section"><a href="trouble.resources.html#trouble.resources.jira">12.3.4. JIRA</a></span></dt></dl></dd><dt><span class="section"><a href="trouble.tools.html">12.4. Tools</a></span></dt><dd><dl><dt><span class="section"><a href="trouble.tools.html#trouble.tools.builtin">12.4.1. Builtin Tools</a></span></dt><dt><span class="section"><a href="trouble.tools.html#trouble.tools.external">12.4.2. External Tools</a></span></dt></dl></dd><dt><span class="section"><a href="trouble.client.html">12.5. Client</a></span></dt><dd><dl><dt><span class="section"><a href="trouble.client.html#tro
uble.client.scantimeout">12.5.1. ScannerTimeoutException or UnknownScannerException</a></span></dt><dt><span class="section"><a href="trouble.client.html#trouble.client.lease.exception">12.5.2. <code class="classname">LeaseException</code> when calling <code class="classname">Scanner.next</code></a></span></dt><dt><span class="section"><a href="trouble.client.html#trouble.client.scarylogs">12.5.3. Shell or client application throws lots of scary exceptions during normal operation</a></span></dt><dt><span class="section"><a href="trouble.client.html#trouble.client.longpauseswithcompression">12.5.4. Long Client Pauses With Compression</a></span></dt><dt><span class="section"><a href="trouble.client.html#trouble.client.zookeeper">12.5.5. ZooKeeper Client Connection Errors</a></span></dt><dt><span class="section"><a href="trouble.client.html#trouble.client.oome.directmemory.leak">12.5.6. Client running out of memory though heap size seems to be stable (but the off-heap/direct he
ap keeps growing)</a></span></dt><dt><span class="section"><a href="trouble.client.html#trouble.client.slowdown.admin">12.5.7. Client Slowdown When Calling Admin Methods (flush, compact, etc.)</a></span></dt><dt><span class="section"><a href="trouble.client.html#trouble.client.security.rpc">12.5.8. Secure Client Cannot Connect ([Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)])</a></span></dt></dl></dd><dt><span class="section"><a href="trouble.mapreduce.html">12.6. MapReduce</a></span></dt><dd><dl><dt><span class="section"><a href="trouble.mapreduce.html#trouble.mapreduce.local">12.6.1. You Think You're On The Cluster, But You're Actually Local</a></span></dt></dl></dd><dt><span class="section"><a href="trouble.namenode.html">12.7. NameNode</a></span></dt><dd><dl><dt><span class="section"><a href="trouble.namenode.html#trouble.namenode.disk">12.7.1. HDFS Utilization of Tables and Regions</a></span></dt><dt><span class
="section"><a href="trouble.namenode.html#trouble.namenode.hbase.objects">12.7.2. Browsing HDFS for HBase Objects</a></span></dt></dl></dd><dt><span class="section"><a href="trouble.network.html">12.8. Network</a></span></dt><dd><dl><dt><span class="section"><a href="trouble.network.html#trouble.network.spikes">12.8.1. Network Spikes</a></span></dt><dt><span class="section"><a href="trouble.network.html#trouble.network.loopback">12.8.2. Loopback IP</a></span></dt><dt><span class="section"><a href="trouble.network.html#trouble.network.ints">12.8.3. Network Interfaces</a></span></dt></dl></dd><dt><span class="section"><a href="trouble.rs.html">12.9. RegionServer</a></span></dt><dd><dl><dt><span class="section"><a href="trouble.rs.html#trouble.rs.startup">12.9.1. Startup Errors</a></span></dt><dt><span class="section"><a href="trouble.rs.html#trouble.rs.runtime">12.9.2. Runtime Errors</a></span></dt><dt><span class="section"><a href="trouble.rs.html#trouble.rs.shutdown">12.9.3.
Shutdown Errors</a></span></dt></dl></dd><dt><span class="section"><a href="trouble.master.html">12.10. Master</a></span></dt><dd><dl><dt><span class="section"><a href="trouble.master.html#trouble.master.startup">12.10.1. Startup Errors</a></span></dt><dt><span class="section"><a href="trouble.master.html#trouble.master.shutdown">12.10.2. Shutdown Errors</a></span></dt></dl></dd><dt><span class="section"><a href="trouble.zookeeper.html">12.11. ZooKeeper</a></span></dt><dd><dl><dt><span class="section"><a href="trouble.zookeeper.html#trouble.zookeeper.startup">12.11.1. Startup Errors</a></span></dt><dt><span class="section"><a href="trouble.zookeeper.html#trouble.zookeeper.general">12.11.2. ZooKeeper, The Cluster Canary</a></span></dt></dl></dd><dt><span class="section"><a href="trouble.ec2.html">12.12. Amazon EC2</a></span></dt><dd><dl><dt><span class="section"><a href="trouble.ec2.html#trouble.ec2.zookeeper">12.12.1. ZooKeeper does not seem to work on Amazon EC2</a></span>
</dt><dt><span class="section"><a href="trouble.ec2.html#trouble.ec2.instability">12.12.2. Instability on Amazon EC2</a></span></dt><dt><span class="section"><a href="trouble.ec2.html#trouble.ec2.connection">12.12.3. Remote Java Connection into EC2 Cluster Not Working</a></span></dt></dl></dd><dt><span class="section"><a href="trouble.versions.html">12.13. HBase and Hadoop version issues</a></span></dt><dd><dl><dt><span class="section"><a href="trouble.versions.html#trouble.versions.205">12.13.1. <code class="code">NoClassDefFoundError</code> when trying to run 0.90.x on hadoop-0.20.205.x (or hadoop-1.0.x)</a></span></dt><dt><span class="section"><a href="trouble.versions.html#trouble.versions.205">12.13.2. ...cannot communicate with client version...</a></span></dt></dl></dd><dt><span class="section"><a href="trouble.casestudy.html">12.14. Case Studies</a></span></dt></dl></div><div class="section" title="12.1. General Guidelines"><div class="titlepage"><div><div><h2 c
lass="title" style="clear: both"><a name="trouble.general"></a>12.1. General Guidelines</h2></div></div></div><p>
Always start with the master log (TODO: Which lines?).
Normally it’s just printing the same lines over and over again.
If not, then there’s an issue.
@@ -39,4 +39,4 @@
dsq.src = 'http://' + disqus_shortname + '.disqus.com/embed.js';
(document.getElementsByTagName('head')[0] || document.getElementsByTagName('body')[0]).appendChild(dsq);
})();
-</script><noscript>Please enable JavaScript to view the <a href="http://disqus.com/?ref_noscript">comments powered by Disqus.</a></noscript><a href="http://disqus.com" class="dsq-brlink">comments powered by <span class="logo-disqus">Disqus</span></a><div class="navfooter"><hr><table width="100%" summary="Navigation footer"><tr><td width="40%" align="left"><a accesskey="p" href="perf.casestudy.html">Prev</a> </td><td width="20%" align="center"> </td><td width="40%" align="right"> <a accesskey="n" href="trouble.log.html">Next</a></td></tr><tr><td width="40%" align="left" valign="top">11.12. Case Studies </td><td width="20%" align="center"><a accesskey="h" href="book.html">Home</a></td><td width="40%" align="right" valign="top"> 12.2. Logs</td></tr></table></div></body></html>
\ No newline at end of file
+</script><noscript>Please enable JavaScript to view the <a href="http://disqus.com/?ref_noscript">comments powered by Disqus.</a></noscript><a href="http://disqus.com" class="dsq-brlink">comments powered by <span class="logo-disqus">Disqus</span></a><div class="navfooter"><hr><table width="100%" summary="Navigation footer"><tr><td width="40%" align="left"><a accesskey="p" href="perf.casestudy.html">Prev</a> </td><td width="20%" align="center"> </td><td width="40%" align="right"> <a accesskey="n" href="trouble.log.html">Next</a></td></tr><tr><td width="40%" align="left" valign="top">11.13. Case Studies </td><td width="20%" align="center"><a accesskey="h" href="book.html">Home</a></td><td width="40%" align="right" valign="top"> 12.2. Logs</td></tr></table></div></body></html>
\ No newline at end of file
Modified: hbase/hbase.apache.org/trunk/book/trouble.resources.html
URL: http://svn.apache.org/viewvc/hbase/hbase.apache.org/trunk/book/trouble.resources.html?rev=1462679&r1=1462678&r2=1462679&view=diff
==============================================================================
--- hbase/hbase.apache.org/trunk/book/trouble.resources.html (original)
+++ hbase/hbase.apache.org/trunk/book/trouble.resources.html Sat Mar 30 00:19:55 2013
@@ -8,12 +8,12 @@
is generally used for questions on released versions of Apache HBase. Before going to the mailing list, make sure your
question has not already been answered by searching the mailing list archives first. Use
<a class="xref" href="trouble.resources.html#trouble.resources.searchhadoop" title="12.3.1. search-hadoop.com">Section 12.3.1, “search-hadoop.com”</a>.
- Take some time crafting your question<sup>[<a name="d2274e7759" href="#ftn.d2274e7759" class="footnote">28</a>]</sup>; a quality question that includes all context and
+ Take some time crafting your question<sup>[<a name="d2279e7773" href="#ftn.d2279e7773" class="footnote">28</a>]</sup>; a quality question that includes all context and
exhibits evidence the author has tried to find answers in the manual and out on lists
is more likely to get a prompt response.
</p></div><div class="section" title="12.3.3. IRC"><div class="titlepage"><div><div><h3 class="title"><a name="trouble.resources.irc"></a>12.3.3. IRC</h3></div></div></div><p>#hbase on irc.freenode.net</p></div><div class="section" title="12.3.4. JIRA"><div class="titlepage"><div><div><h3 class="title"><a name="trouble.resources.jira"></a>12.3.4. JIRA</h3></div></div></div><p>
<a class="link" href="https://issues.apache.org/jira/browse/HBASE" target="_top">JIRA</a> is also really helpful when looking for Hadoop/HBase-specific issues.
- </p></div><div class="footnotes"><br><hr width="100" align="left"><div class="footnote"><p><sup>[<a id="ftn.d2274e7759" href="#d2274e7759" class="para">28</a>] </sup>See Getting Answers</p></div></div></div><div id="disqus_thread"></div><script type="text/javascript">
+ </p></div><div class="footnotes"><br><hr width="100" align="left"><div class="footnote"><p><sup>[<a id="ftn.d2279e7773" href="#d2279e7773" class="para">28</a>] </sup>See Getting Answers</p></div></div></div><div id="disqus_thread"></div><script type="text/javascript">
var disqus_shortname = 'hbase'; // required: replace example with your forum shortname
var disqus_url = 'http://hbase.apache.org/book';
var disqus_identifier = 'trouble.resources';
Modified: hbase/hbase.apache.org/trunk/book/upgrade0.90.html
URL: http://svn.apache.org/viewvc/hbase/hbase.apache.org/trunk/book/upgrade0.90.html?rev=1462679&r1=1462678&r2=1462679&view=diff
==============================================================================
--- hbase/hbase.apache.org/trunk/book/upgrade0.90.html (original)
+++ hbase/hbase.apache.org/trunk/book/upgrade0.90.html Sat Mar 30 00:19:55 2013
@@ -27,10 +27,10 @@
need to change this (The 'normal'/default value is 64MB (67108864)).
Run the script <code class="filename">bin/set_meta_memstore_size.rb</code>.
This will make the necessary edit to your <code class="varname">.META.</code> schema.
- Failure to run this change will make for a slow cluster <sup>[<a name="d2274e2863" href="#ftn.d2274e2863" class="footnote">12</a>]</sup>
+ Failure to run this change will make for a slow cluster <sup>[<a name="d2279e2863" href="#ftn.d2279e2863" class="footnote">12</a>]</sup>
.
- </p><div class="footnotes"><br><hr width="100" align="left"><div class="footnote"><p><sup>[<a id="ftn.d2274e2863" href="#d2274e2863" class="para">12</a>] </sup>
+ </p><div class="footnotes"><br><hr width="100" align="left"><div class="footnote"><p><sup>[<a id="ftn.d2279e2863" href="#d2279e2863" class="para">12</a>] </sup>
See <a class="link" href="https://issues.apache.org/jira/browse/HBASE-3499" target="_top">HBASE-3499 Users upgrading to 0.90.0 need to have their .META. table updated with the right MEMSTORE_SIZE</a>
</p></div></div></div><div id="disqus_thread"></div><script type="text/javascript">
var disqus_shortname = 'hbase'; // required: replace example with your forum shortname
Modified: hbase/hbase.apache.org/trunk/book/upgrade0.92.html
URL: http://svn.apache.org/viewvc/hbase/hbase.apache.org/trunk/book/upgrade0.92.html?rev=1462679&r1=1462678&r2=1462679&view=diff
==============================================================================
--- hbase/hbase.apache.org/trunk/book/upgrade0.92.html (original)
+++ hbase/hbase.apache.org/trunk/book/upgrade0.92.html Sat Mar 30 00:19:55 2013
@@ -14,46 +14,46 @@ There’s a separate tarball for sec
If -XX:MaxDirectMemorySize is set in your hbase-env.sh, it’s going to enable the experimental off-heap cache (You may not want this).
</li></ol></div><p>
</p></div><p>
-</p><div class="section" title="3.3.1. You can’t go back!"><div class="titlepage"><div><div><h3 class="title"><a name="d2274e2710"></a>3.3.1. You can’t go back!
+</p><div class="section" title="3.3.1. You can’t go back!"><div class="titlepage"><div><div><h3 class="title"><a name="d2279e2710"></a>3.3.1. You can’t go back!
</h3></div></div></div><p>To move to 0.92.0, all you need to do is shutdown your cluster, replace your hbase 0.90.x with hbase 0.92.0 binaries (be sure you clear out all 0.90.x instances) and restart (You cannot do a rolling restart from 0.90.x to 0.92.x -- you must restart).
On startup, the <code class="varname">.META.</code> table content is rewritten removing the table schema from the <code class="varname">info:regioninfo</code> column.
Also, any flushes done post first startup will write out data in the new 0.92.0 file format, <a class="link" href="http://hbase.apache.org/book.html#hfilev2" target="_top">HFile V2</a>.
This means you cannot go back to 0.90.x once you’ve started HBase 0.92.0 over your HBase data directory.
-</p></div><div class="section" title="3.3.2. MSLAB is ON by default"><div class="titlepage"><div><div><h3 class="title"><a name="d2274e2724"></a>3.3.2. MSLAB is ON by default
+</p></div><div class="section" title="3.3.2. MSLAB is ON by default"><div class="titlepage"><div><div><h3 class="title"><a name="d2279e2724"></a>3.3.2. MSLAB is ON by default
</h3></div></div></div><p>In 0.92.0, the <a class="link" href="http://hbase.apache.org/book.html#hbase.hregion.memstore.mslab.enabled" target="_top">hbase.hregion.memstore.mslab.enabled</a> flag is set to true
(See <a class="xref" href="jvm.html#mslab">Section 11.3.1.1, “Long GC pauses”</a>). In 0.90.x it was <code class="constant">false</code>. When it is enabled, memstores will step allocate memory in MSLAB 2MB chunks even if the
memstore has zero or just a few small elements. This is fine usually but if you had lots of regions per regionserver in a 0.90.x cluster (and MSLAB was off),
you may find yourself OOME'ing on upgrade because the <span class="mathphrase">thousands of regions * number of column families * 2MB MSLAB (at a minimum)</span>
puts your heap over the top. Set <code class="varname">hbase.hregion.memstore.mslab.enabled</code> to
<code class="constant">false</code> or set the MSLAB size down from 2MB by setting <code class="varname">hbase.hregion.memstore.mslab.chunksize</code> to something less.
-</p></div><div class="section" title="3.3.3. Distributed splitting is on by default"><div class="titlepage"><div><div><h3 class="title"><a name="d2274e2749"></a>3.3.3. Distributed splitting is on by default
+</p></div><div class="section" title="3.3.3. Distributed splitting is on by default"><div class="titlepage"><div><div><h3 class="title"><a name="d2279e2749"></a>3.3.3. Distributed splitting is on by default
</h3></div></div></div><p>Previous, WAL logs on crash were split by the Master alone. In 0.92.0, log splitting is done by the cluster (See See “HBASE-1364 [performance] Distributed splitting of regionserver commit logs”). This should cut down significantly on the amount of time it takes splitting logs and getting regions back online again.
-</p></div><div class="section" title="3.3.4. Memory accounting is different now"><div class="titlepage"><div><div><h3 class="title"><a name="d2274e2754"></a>3.3.4. Memory accounting is different now
+</p></div><div class="section" title="3.3.4. Memory accounting is different now"><div class="titlepage"><div><div><h3 class="title"><a name="d2279e2754"></a>3.3.4. Memory accounting is different now
</h3></div></div></div><p>In 0.92.0, <a class="xref" href="hfilev2.html" title="Appendix E. HFile format version 2">Appendix E, <i>HFile format version 2</i></a> indices and bloom filters take up residence in the same LRU used caching blocks that come from the filesystem.
In 0.90.x, the HFile v1 indices lived outside of the LRU so they took up space even if the index was on a ‘cold’ file, one that wasn’t being actively used. With the indices now in the LRU, you may find you
have less space for block caching. Adjust your block cache accordingly. See the <a class="xref" href="regionserver.arch.html#block.cache" title="9.6.4. Block Cache">Section 9.6.4, “Block Cache”</a> for more detail.
The block size default size has been changed in 0.92.0 from 0.2 (20 percent of heap) to 0.25.
-</p></div><div class="section" title="3.3.5. On the Hadoop version to use"><div class="titlepage"><div><div><h3 class="title"><a name="d2274e2763"></a>3.3.5. On the Hadoop version to use
+</p></div><div class="section" title="3.3.5. On the Hadoop version to use"><div class="titlepage"><div><div><h3 class="title"><a name="d2279e2763"></a>3.3.5. On the Hadoop version to use
</h3></div></div></div><p>Run 0.92.0 on Hadoop 1.0.x (or CDH3u3 when it ships). The performance benefits are worth making the move. Otherwise, our Hadoop prescription is as it has been; you need an Hadoop that supports a working sync. See <a class="xref" href="configuration.html#hadoop" title="2.1.3. Hadoop">Section 2.1.3, “Hadoop”</a>.
</p><p>If running on Hadoop 1.0.x (or CDH3u3), enable local read. See <a class="link" href="http://files.meetup.com/1350427/hug_ebay_jdcryans.pdf" target="_top">Practical Caching</a> presentation for ruminations on the performance benefits ‘going local’ (and for how to enable local reads).
-</p></div><div class="section" title="3.3.6. HBase 0.92.0 ships with ZooKeeper 3.4.2"><div class="titlepage"><div><div><h3 class="title"><a name="d2274e2775"></a>3.3.6. HBase 0.92.0 ships with ZooKeeper 3.4.2
+</p></div><div class="section" title="3.3.6. HBase 0.92.0 ships with ZooKeeper 3.4.2"><div class="titlepage"><div><div><h3 class="title"><a name="d2279e2775"></a>3.3.6. HBase 0.92.0 ships with ZooKeeper 3.4.2
</h3></div></div></div><p>If you can, upgrade your zookeeper. If you can’t, 3.4.2 clients should work against 3.3.X ensembles (HBase makes use of 3.4.2 API).
-</p></div><div class="section" title="3.3.7. Online alter is off by default"><div class="titlepage"><div><div><h3 class="title"><a name="d2274e2780"></a>3.3.7. Online alter is off by default
+</p></div><div class="section" title="3.3.7. Online alter is off by default"><div class="titlepage"><div><div><h3 class="title"><a name="d2279e2780"></a>3.3.7. Online alter is off by default
</h3></div></div></div><p>In 0.92.0, we’ve added an experimental online schema alter facility (See <a class="xref" href="config.files.html#hbase.online.schema.update.enable" title="hbase.online.schema.update.enable"><code class="varname">hbase.online.schema.update.enable</code></a>). Its off by default. Enable it at your own risk. Online alter and splitting tables do not play well together so be sure your cluster quiescent using this feature (for now).
-</p></div><div class="section" title="3.3.8. WebUI"><div class="titlepage"><div><div><h3 class="title"><a name="d2274e2787"></a>3.3.8. WebUI
+</p></div><div class="section" title="3.3.8. WebUI"><div class="titlepage"><div><div><h3 class="title"><a name="d2279e2787"></a>3.3.8. WebUI
</h3></div></div></div><p>The webui has had a few additions made in 0.92.0. It now shows a list of the regions currently transitioning, recent compactions/flushes, and a process list of running processes (usually empty if all is well and requests are being handled promptly). Other additions including requests by region, a debugging servlet dump, etc.
-</p></div><div class="section" title="3.3.9. Security tarball"><div class="titlepage"><div><div><h3 class="title"><a name="d2274e2792"></a>3.3.9. Security tarball
+</p></div><div class="section" title="3.3.9. Security tarball"><div class="titlepage"><div><div><h3 class="title"><a name="d2279e2792"></a>3.3.9. Security tarball
</h3></div></div></div><p>We now ship with two tarballs; secure and insecure HBase. Documentation on how to setup a secure HBase is on the way.
-</p></div><div class="section" title="3.3.10. Experimental off-heap cache"><div class="titlepage"><div><div><h3 class="title"><a name="d2274e2797"></a>3.3.10. Experimental off-heap cache
+</p></div><div class="section" title="3.3.10. Experimental off-heap cache"><div class="titlepage"><div><div><h3 class="title"><a name="d2279e2797"></a>3.3.10. Experimental off-heap cache
</h3></div></div></div><p>
A new cache was contributed to 0.92.0 to act as a solution between using the “on-heap” cache which is the current LRU cache the region servers have and the operating system cache which is out of our control.
To enable, set “-XX:MaxDirectMemorySize” in hbase-env.sh to the value for maximum direct memory size and specify hbase.offheapcache.percentage in hbase-site.xml with the percentage that you want to dedicate to off-heap cache. This should only be set for servers and not for clients. Use at your own risk.
See this blog post for additional information on this new experimental feature: http://www.cloudera.com/blog/2012/01/caching-in-hbase-slabcache/
-</p></div><div class="section" title="3.3.11. Changes in HBase replication"><div class="titlepage"><div><div><h3 class="title"><a name="d2274e2802"></a>3.3.11. Changes in HBase replication
+</p></div><div class="section" title="3.3.11. Changes in HBase replication"><div class="titlepage"><div><div><h3 class="title"><a name="d2279e2802"></a>3.3.11. Changes in HBase replication
</h3></div></div></div><p>0.92.0 adds two new features: multi-slave and multi-master replication. The way to enable this is the same as adding a new peer, so in order to have multi-master you would just run add_peer for each cluster that acts as a master to the other slave clusters. Collisions are handled at the timestamp level which may or may not be what you want, this needs to be evaluated on a per use case basis. Replication is still experimental in 0.92 and is disabled by default, run it at your own risk.
-</p></div><div class="section" title="3.3.12. RegionServer now aborts if OOME"><div class="titlepage"><div><div><h3 class="title"><a name="d2274e2807"></a>3.3.12. RegionServer now aborts if OOME
+</p></div><div class="section" title="3.3.12. RegionServer now aborts if OOME"><div class="titlepage"><div><div><h3 class="title"><a name="d2279e2807"></a>3.3.12. RegionServer now aborts if OOME
</h3></div></div></div><p>If an OOME, we now have the JVM kill -9 the regionserver process so it goes down fast. Previous, a RegionServer might stick around after incurring an OOME limping along in some wounded state. To disable this facility, and recommend you leave it in place, you’d need to edit the bin/hbase file. Look for the addition of the -XX:OnOutOfMemoryError="kill -9 %p" arguments (See [HBASE-4769] - ‘Abort RegionServer Immediately on OOME’)
-</p></div><div class="section" title="3.3.13. HFile V2 and the “Bigger, Fewer” Tendency"><div class="titlepage"><div><div><h3 class="title"><a name="d2274e2812"></a>3.3.13. HFile V2 and the “Bigger, Fewer” Tendency
+</p></div><div class="section" title="3.3.13. HFile V2 and the “Bigger, Fewer” Tendency"><div class="titlepage"><div><div><h3 class="title"><a name="d2279e2812"></a>3.3.13. HFile V2 and the “Bigger, Fewer” Tendency
</h3></div></div></div><p>0.92.0 stores data in a new format, <a class="xref" href="hfilev2.html" title="Appendix E. HFile format version 2">Appendix E, <i>HFile format version 2</i></a>. As HBase runs, it will move all your data from HFile v1 to HFile v2 format. This auto-migration will run in the background as flushes and compactions run.
HFile V2 allows HBase run with larger regions/files. In fact, we encourage that all HBasers going forward tend toward Facebook axiom #1, run with larger, fewer regions.
If you have lots of regions now -- more than 100s per host -- you should look into setting your region size up after you move to 0.92.0 (In 0.92.0, default size is now 1G, up from 256M), and then running online merge tool (See “HBASE-1621 merge tool should work on online cluster, but disabled table”).