You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@phoenix.apache.org by je...@apache.org on 2014/08/24 01:24:32 UTC
svn commit: r1620094 - in /phoenix/site: publish/secondary_indexing.html
publish/tuning.html source/src/site/markdown/secondary_indexing.md
source/src/site/markdown/tuning.md
Author: jeffreyz
Date: Sat Aug 23 23:24:32 2014
New Revision: 1620094
URL: http://svn.apache.org/r1620094
Log:
PHOENIX-1158: Update secondary index failure policy documentation
Modified:
phoenix/site/publish/secondary_indexing.html
phoenix/site/publish/tuning.html
phoenix/site/source/src/site/markdown/secondary_indexing.md
phoenix/site/source/src/site/markdown/tuning.md
Modified: phoenix/site/publish/secondary_indexing.html
URL: http://svn.apache.org/viewvc/phoenix/site/publish/secondary_indexing.html?rev=1620094&r1=1620093&r2=1620094&view=diff
==============================================================================
--- phoenix/site/publish/secondary_indexing.html (original)
+++ phoenix/site/publish/secondary_indexing.html Sat Aug 23 23:24:32 2014
@@ -1,7 +1,7 @@
<!DOCTYPE html>
<!--
- Generated by Apache Maven Doxia at 2014-08-18
+ Generated by Apache Maven Doxia at 2014-08-23
Rendered using Reflow Maven Skin 1.1.0 (http://andriusvelykis.github.io/reflow-maven-skin)
-->
<html xml:lang="en" lang="en">
@@ -216,14 +216,8 @@
</div>
<div class="section">
<h3 id="Failure_Policy">Failure Policy</h3>
- <p>In the event that the region server handling the data updates cannot write to the region server handling the index updates, the index is automatically disabled and will no longer be considered for use in queries (as it will no longer be in sync with the data table). To use it again, it must be manually rebuilt with the following command:</p>
- <div class="source">
- <pre>ALTER INDEX my_index ON my_table REBUILD;
-</pre>
- </div>
+ <p>In the event that the region server handling the data updates cannot write to the region server handling the index updates, the index will be automatically disabled temporally and will no longer be considered for use in queries (as it will no longer be in sync with the data table). There is a backend job will soon rebuild the index from where it failed before as soon as all regions of the underlying index table are online. A user can turn off this auto rebuild index behavior by setting configuration âphoenix.index.failure.handling.rebuildâ to false in hbase-site.xml on each region server.</p>
<p>If we cannot disable the index, then the server will be immediately aborted. If the abort fails, we call System.exit on the JVM, forcing the server to die. By killing the server, we ensure that the WAL will be replayed on recovery, replaying the index updates to their appropriate tables.</p>
- <p><b>WARNING: global indexing has the potential to bring down your entire cluster very quickly.</b></p>
- <p>If the index tables are not setup correctly (Phoenix ensures that they are), this failure policy can cause a cascading failure as each region server attempts and fails to write the index update, subsequently killing itself to ensure the visibility concerns outlined above.</p>
</div>
</div>
<div class="section">
@@ -333,6 +327,7 @@
<li><a class="externalLink" href="http://www.slideshare.net/jesse_yates/phoenix-secondary-indexing-la-hug-sept-9th-2013">Los Anglees HBase Meetup</a> - Sept, 4th, 2013</li>
<li><a class="externalLink" href="https://github.com/Huawei-Hadoop/hindex/blob/master/README.md#how-it-works">Local Indexes</a> by Huawei</li>
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/PHOENIX-938">PHOENIX-938</a> and <a class="externalLink" href="https://issues.apache.org/jira/browse/HBASE-11513">HBASE-11513</a> for deadlock prevention during global index maintenance.</li>
+ <li><a class="externalLink" href="https://issues.apache.org/jira/browse/PHOENIX-1112">PHOENIX-1112: Atomically rebuild index partially when index update fails</a></li>
</ul>
</div>
</div>
Modified: phoenix/site/publish/tuning.html
URL: http://svn.apache.org/viewvc/phoenix/site/publish/tuning.html?rev=1620094&r1=1620093&r2=1620094&view=diff
==============================================================================
--- phoenix/site/publish/tuning.html (original)
+++ phoenix/site/publish/tuning.html Sat Aug 23 23:24:32 2014
@@ -1,7 +1,7 @@
<!DOCTYPE html>
<!--
- Generated by Apache Maven Doxia at 2014-08-18
+ Generated by Apache Maven Doxia at 2014-08-23
Rendered using Reflow Maven Skin 1.1.0 (http://andriusvelykis.github.io/reflow-maven-skin)
-->
<html xml:lang="en" lang="en">
@@ -317,6 +317,21 @@
<td style="text-align: left;">Delay interval(in milliseconds) when opening SYSTEM.CATALOG to compensate possible time clock skew when SYSTEM.CATALOG moves among region servers. </td>
<td>2000</td>
</tr>
+ <tr class="a">
+ <td><small>phoenix.index.failure.handling.rebuild</small></td>
+ <td style="text-align: left;">Boolean flag which turns on/off auto-rebuild a failed index from when some updates are failed to be updated into the index. </td>
+ <td>true</td>
+ </tr>
+ <tr class="b">
+ <td><small>phoenix.index.failure.handling.rebuild.interval</small></td>
+ <td style="text-align: left;">Time interval(in milliseconds) for index rebuild backend Job to check if there is an index to be rebuilt </td>
+ <td>10000</td>
+ </tr>
+ <tr class="a">
+ <td><small>phoenix.index.failure.handling.rebuild.overlap.time</small></td>
+ <td style="text-align: left;">Index rebuild job builds an index from when it failed - the time interval(in milliseconds) in order to create a time overlap to prevent missing updates when there exists time clock skew. </td>
+ <td>300000</td>
+ </tr>
</tbody>
</table>
<br />
Modified: phoenix/site/source/src/site/markdown/secondary_indexing.md
URL: http://svn.apache.org/viewvc/phoenix/site/source/src/site/markdown/secondary_indexing.md?rev=1620094&r1=1620093&r2=1620094&view=diff
==============================================================================
--- phoenix/site/source/src/site/markdown/secondary_indexing.md (original)
+++ phoenix/site/source/src/site/markdown/secondary_indexing.md Sat Aug 23 23:24:32 2014
@@ -90,17 +90,10 @@ Once the WAL is written, we ensure that
### Failure Policy
-In the event that the region server handling the data updates cannot write to the region server handling the index updates, the index is automatically disabled and will no longer be considered for use in queries (as it will no longer be in sync with the data table). To use it again, it must be manually rebuilt with the following command:
-
-```
-ALTER INDEX my_index ON my_table REBUILD;
-```
+In the event that the region server handling the data updates cannot write to the region server handling the index updates, the index will be automatically disabled temporally and will no longer be considered for use in queries (as it will no longer be in sync with the data table). There is a backend job will soon rebuild the index from where it failed before as soon as all regions of the underlying index table are online. A user can turn off this auto rebuild index behavior by setting configuration âphoenix.index.failure.handling.rebuildâ to false in hbase-site.xml on each region server.
If we cannot disable the index, then the server will be immediately aborted. If the abort fails, we call System.exit on the JVM, forcing the server to die. By killing the server, we ensure that the WAL will be replayed on recovery, replaying the index updates to their appropriate tables.
-**WARNING: global indexing has the potential to bring down your entire cluster very quickly.**
-
-If the index tables are not setup correctly (Phoenix ensures that they are), this failure policy can cause a cascading failure as each region server attempts and fails to write the index update, subsequently killing itself to ensure the visibility concerns outlined above.
## Setup
@@ -197,3 +190,4 @@ There have been several presentations gi
* [Los Anglees HBase Meetup](http://www.slideshare.net/jesse_yates/phoenix-secondary-indexing-la-hug-sept-9th-2013) - Sept, 4th, 2013
* [Local Indexes](https://github.com/Huawei-Hadoop/hindex/blob/master/README.md#how-it-works) by Huawei
* [PHOENIX-938](https://issues.apache.org/jira/browse/PHOENIX-938) and [HBASE-11513](https://issues.apache.org/jira/browse/HBASE-11513) for deadlock prevention during global index maintenance.
+* [PHOENIX-1112: Atomically rebuild index partially when index update fails](https://issues.apache.org/jira/browse/PHOENIX-1112)
Modified: phoenix/site/source/src/site/markdown/tuning.md
URL: http://svn.apache.org/viewvc/phoenix/site/source/src/site/markdown/tuning.md?rev=1620094&r1=1620093&r2=1620094&view=diff
==============================================================================
--- phoenix/site/source/src/site/markdown/tuning.md (original)
+++ phoenix/site/source/src/site/markdown/tuning.md Sat Aug 23 23:24:32 2014
@@ -109,6 +109,12 @@ overridden at connection
</td><td> </td></tr>
<tr><td><small>phoenix.clock.skew.interval</small></td><td style="text-align: left;">Delay interval(in milliseconds) when opening SYSTEM.CATALOG to compensate possible time clock skew when SYSTEM.CATALOG moves among region servers.
</td><td>2000</td></tr>
+<tr><td><small>phoenix.index.failure.handling.rebuild</small></td><td style="text-align: left;">Boolean flag which turns on/off auto-rebuild a failed index from when some updates are failed to be updated into the index.
+</td><td>true</td></tr>
+<tr><td><small>phoenix.index.failure.handling.rebuild.interval</small></td><td style="text-align: left;">Time interval(in milliseconds) for index rebuild backend Job to check if there is an index to be rebuilt
+</td><td>10000</td></tr>
+<tr><td><small>phoenix.index.failure.handling.rebuild.overlap.time</small></td><td style="text-align: left;">Index rebuild job builds an index from when it failed - the time interval(in milliseconds) in order to create a time overlap to prevent missing updates when there exists time clock skew.
+</td><td>300000</td></tr>
</tbody></table>
<br />
<h4>