You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hbase.apache.org by gi...@apache.org on 2018/08/02 19:52:35 UTC
[49/51] [partial] hbase-site git commit: Published site at
613d831429960348dc42c3bdb6ea5d31be15c81c.
http://git-wip-us.apache.org/repos/asf/hbase-site/blob/7cf6034b/book.html
----------------------------------------------------------------------
diff --git a/book.html b/book.html
index fc5ffdb..4c27d13 100644
--- a/book.html
+++ b/book.html
@@ -14672,6 +14672,121 @@ See <a href="#arch.catalog.meta">hbase:meta</a> for more information on the meta
</div>
</div>
</div>
+<div class="sect2">
+<h3 id="master.wal"><a class="anchor" href="#master.wal"></a>69.5. MasterProcWAL</h3>
+<div class="paragraph">
+<p>HMaster records administrative operations and their running states, such as the handling of a crashed server,
+table creation, and other DDLs, into its own WAL file. The WALs are stored under the MasterProcWALs
+directory. The Master WALs are not like RegionServer WALs. Keeping up the Master WAL allows
+us run a state machine that is resilient across Master failures. For example, if a HMaster was in the
+middle of creating a table encounters an issue and fails, the next active HMaster can take up where
+the previous left off and carry the operation to completion. Since hbase-2.0.0, a
+new AssignmentManager (A.K.A AMv2) was introduced and the HMaster handles region assignment
+operations, server crash processing, balancing, etc., all via AMv2 persisting all state and
+transitions into MasterProcWALs rather than up into ZooKeeper, as we do in hbase-1.x.</p>
+</div>
+<div class="paragraph">
+<p>See <a href="#amv2">AMv2 Description for Devs</a> (and <a href="#pv2">Procedure Framework (Pv2): <a href="https://issues.apache.org/jira/browse/HBASE-12439">HBASE-12439</a></a> for its basis) if you would like to learn more about the new
+AssignmentManager.</p>
+</div>
+<div class="sect3">
+<h4 id="master.wal.conf"><a class="anchor" href="#master.wal.conf"></a>69.5.1. Configurations for MasterProcWAL</h4>
+<div class="paragraph">
+<p>Here are the list of configurations that effect MasterProcWAL operation.
+You should not have to change your defaults.</p>
+</div>
+<div id="hbase.procedure.store.wal.periodic.roll.msec" class="dlist">
+<dl>
+<dt class="hdlist1"><strong><code>hbase.procedure.store.wal.periodic.roll.msec</code></strong></dt>
+<dd>
+<div class="paragraph">
+<div class="title">Description</div>
+<p>Frequency of generating a new WAL</p>
+</div>
+<div class="paragraph">
+<div class="title">Default</div>
+<p><code>1h (3600000 in msec)</code></p>
+</div>
+</dd>
+</dl>
+</div>
+<div id="hbase.procedure.store.wal.roll.threshold" class="dlist">
+<dl>
+<dt class="hdlist1"><strong><code>hbase.procedure.store.wal.roll.threshold</code></strong></dt>
+<dd>
+<div class="paragraph">
+<div class="title">Description</div>
+<p>Threshold in size before the WAL rolls. Every time the WAL reaches this size or the above period, 1 hour, passes since last log roll, the HMaster will generate a new WAL.</p>
+</div>
+<div class="paragraph">
+<div class="title">Default</div>
+<p><code>32MB (33554432 in byte)</code></p>
+</div>
+</dd>
+</dl>
+</div>
+<div id="hbase.procedure.store.wal.warn.threshold" class="dlist">
+<dl>
+<dt class="hdlist1"><strong><code>hbase.procedure.store.wal.warn.threshold</code></strong></dt>
+<dd>
+<div class="paragraph">
+<div class="title">Description</div>
+<p>If the number of WALs goes beyond this threshold, the following message should appear in the HMaster log with WARN level when rolling.</p>
+</div>
+<div class="literalblock">
+<div class="content">
+<pre>procedure WALs count=xx above the warning threshold 64. check running procedures to see if something is stuck.</pre>
+</div>
+</div>
+<div class="paragraph">
+<div class="title">Default</div>
+<p><code>64</code></p>
+</div>
+</dd>
+</dl>
+</div>
+<div id="hbase.procedure.store.wal.max.retries.before.roll" class="dlist">
+<dl>
+<dt class="hdlist1"><strong><code>hbase.procedure.store.wal.max.retries.before.roll</code></strong></dt>
+<dd>
+<div class="paragraph">
+<div class="title">Description</div>
+<p>Max number of retry when syncing slots (records) to its underlying storage, such as HDFS. Every attempt, the following message should appear in the HMaster log.</p>
+</div>
+<div class="literalblock">
+<div class="content">
+<pre>unable to sync slots, retry=xx</pre>
+</div>
+</div>
+<div class="paragraph">
+<div class="title">Default</div>
+<p><code>3</code></p>
+</div>
+</dd>
+</dl>
+</div>
+<div id="hbase.procedure.store.wal.sync.failure.roll.max" class="dlist">
+<dl>
+<dt class="hdlist1"><strong><code>hbase.procedure.store.wal.sync.failure.roll.max</code></strong></dt>
+<dd>
+<div class="paragraph">
+<div class="title">Description</div>
+<p>After the above 3 retrials, the log is rolled and the retry count is reset to 0, thereon a new set of retrial starts. This configuration controls the max number of attempts of log rolling upon sync failure. That is, HMaster is allowed to fail to sync 9 times in total. Once it exceeds, the following log should appear in the HMaster log.</p>
+</div>
+<div class="literalblock">
+<div class="content">
+<pre>Sync slots after log roll failed, abort.</pre>
+</div>
+</div>
+<div class="paragraph">
+<div class="title">Default</div>
+<p><code>3</code></p>
+</div>
+</dd>
+</dl>
+</div>
+</div>
+</div>
</div>
</div>
<div class="sect1">
@@ -15320,7 +15435,8 @@ You will likely find references to the HLog in documentation tailored to these o
by a short name label (that unfortunately is not always descriptive). You set the provider in
<em>hbase-site.xml</em> passing the WAL provder short-name as the value on the
<em>hbase.wal.provider</em> property (Set the provider for <em>hbase:meta</em> using the
-<em>hbase.wal.meta_provider</em> property).</p>
+<em>hbase.wal.meta_provider</em> property, otherwise it uses the same provider configured
+by <em>hbase.wal.provider</em>).</p>
</div>
<div class="ulist">
<ul>
@@ -40976,7 +41092,7 @@ org/apache/hadoop/hbase/security/access/AccessControlClient.revoke:(Lorg/apache/
<div id="footer">
<div id="footer-text">
Version 3.0.0-SNAPSHOT<br>
-Last updated 2018-08-01 14:29:55 UTC
+Last updated 2018-08-02 19:32:10 UTC
</div>
</div>
</body>
http://git-wip-us.apache.org/repos/asf/hbase-site/blob/7cf6034b/bulk-loads.html
----------------------------------------------------------------------
diff --git a/bulk-loads.html b/bulk-loads.html
index 9b3e7dd..e292b41 100644
--- a/bulk-loads.html
+++ b/bulk-loads.html
@@ -7,7 +7,7 @@
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20180801" />
+ <meta name="Date-Revision-yyyymmdd" content="20180802" />
<meta http-equiv="Content-Language" content="en" />
<title>Apache HBase –
Bulk Loads in Apache HBase (TM)
@@ -306,7 +306,7 @@ under the License. -->
<a href="https://www.apache.org/">The Apache Software Foundation</a>.
All rights reserved.
- <li id="publishDate" class="pull-right">Last Published: 2018-08-01</li>
+ <li id="publishDate" class="pull-right">Last Published: 2018-08-02</li>
</p>
</div>