You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@omid.apache.org by oh...@apache.org on 2018/09/26 09:18:44 UTC
[44/50] incubator-omid-site git commit: [wagon-git] commit to branch
asf-site 9/26/18 12:17 PM
http://git-wip-us.apache.org/repos/asf/incubator-omid-site/blob/ff061775/quickstart.html
----------------------------------------------------------------------
diff --git a/quickstart.html b/quickstart.html
index 5183e10..f6b9777 100644
--- a/quickstart.html
+++ b/quickstart.html
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia Site Renderer 1.8.1 at 2018-09-26
+ | Generated by Apache Maven Doxia Site Renderer 1.7 at 2018-09-20
| Rendered using Apache Maven Fluido Skin 1.5
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20180926" />
+ <meta name="Date-Revision-yyyymmdd" content="20180920" />
<meta http-equiv="Content-Language" content="en" />
<title>Apache Omid – Quickstart</title>
<link rel="stylesheet" href="./css/apache-maven-fluido-1.5.min.css" />
@@ -117,7 +117,7 @@
</li>
<li> <a href="index.html" title="About">About</a>
</li>
- <li> <a href="licenses.html" title="Licenses">Licenses</a>
+ <li> <a href="license.html" title="Licenses">Licenses</a>
</li>
<li> <a href="modules.html" title="Project Modules">Project Modules</a>
</li>
@@ -125,11 +125,11 @@
</li>
<li> <a href="plugins.html" title="Plugins">Plugins</a>
</li>
- <li> <a href="scm.html" title="Source Code Management">Source Code Management</a>
+ <li> <a href="team-list.html" title="Team">Team</a>
</li>
- <li> <a href="summary.html" title="Summary">Summary</a>
+ <li> <a href="source-repository.html" title="Source Code Management">Source Code Management</a>
</li>
- <li> <a href="team.html" title="Team">Team</a>
+ <li> <a href="project-summary.html" title="Summary">Summary</a>
</li>
</ul>
</li>
@@ -194,11 +194,11 @@
<div id="breadcrumbs">
<ul class="breadcrumb">
- <li id="projectVersion">Version: 0.8.2.11-SNAPSHOT
+ <li id="projectVersion">Version: 0.9.0.0
</li>
- <li id="publishDate" class="pull-right">Last Published: 2018-09-26</li>
+ <li id="publishDate" class="pull-right">Last Published: 2018-09-20</li>
</ul>
</div>
@@ -207,15 +207,31 @@
<div id="bodyColumn" >
- <h1>Quickstart</h1>
+ <!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+--><h1>Quickstart</h1>
<p>Below are instructions to quickly set up an environment to test Omid in your local machine.</p>
<div class="section">
<h2><a name="Requirements"></a>Requirements</h2>
-<ol style="list-style-type: decimal">
+<ol style="list-style-type: decimal">
+
<li>Maven 3.x</li>
+
<li>Java 7</li>
+
<li>HBase 0.98</li>
+
<li>Protobuf 2.5.0</li>
</ol></div>
<div class="section">
@@ -226,47 +242,39 @@
<div class="section">
<h3><a name="a2._Clone_the_Omid_repository_and_Build_the_TSO_Package:"></a>2. Clone the <a class="externalLink" href="https://github.com/apache/incubator-omid">Omid repository</a> and Build the TSO Package:</h3>
-<div>
-<div>
-<pre class="source">$ git clone git@github.com:apache/incubator-omid.git
+<div class="source">
+<div class="source"><pre class="prettyprint">$ git clone git@github.com:yahoo/omid.git
$ cd omid
-$ mvn clean install -Phbase-1 (for HBase 1.x versions)
+$ mvn clean install -Phbase-0 (for HBase 0.x versions)
or
-$ mvn clean install -Phbase-2 (for HBase 2.x versions)
+$ mvn clean install -Phbase-1 (for HBase 1.x versions)
</pre></div></div>
-
-<p>This will generate a binary package containing all dependencies for the TSO in tso-server/target/tso-server-<VERSION>-bin.tar.gz.</p>
+<p>This will generate a binary package containing all dependencies for the TSO in tso-server/target/tso-server-<VERSION>-bin.tar.gz. </p>
<p><b>Be aware</b> Unit tests use HBase mini cluster, it typically fails to start if you are on VPN, thus unit test fail. Unit tests coverage is also quite extensive and take a while to run on each build (~15min at the moment of writing). So, consider using <tt>mvn clean install -DskipTests</tt> to speed temporal builds. Note that <tt>-Dmaven.test.skip=true</tt> <a class="externalLink" href="http://ericlefevre.net/wordpress/2008/02/21/skipping-tests-with-maven/">is NOT an equivalent</a>.</p>
<p>As an alternative to clone the project, you can download the required version for the TSO tar.gz package from the <a class="externalLink" href="https://dist.apache.org/repos/dist/release/incubator/omid/">release repository</a>.</p>
<p>You can also see the <a class="externalLink" href="https://github.com/apache/incubator-omid/tags">build history here</a>.</p></div>
<div class="section">
<h3><a name="a3._Extract_the_TSO_Package"></a>3. Extract the TSO Package</h3>
-<div>
-<div>
-<pre class="source">$ tar zxvf tso-server-<VERSION>-bin.tar.gz
+<div class="source">
+<div class="source"><pre class="prettyprint">$ tar zxvf tso-server-<VERSION>-bin.tar.gz
$ cd tso-server-<VERSION>
-</pre></div></div>
-</div>
+</pre></div></div></div>
<div class="section">
<h3><a name="a4._Create_Omid_Tables"></a>4. Create Omid Tables</h3>
<p>Ensure that the setting for hbase.zookeeper.quorum in conf/hbase-site.xml points to your zookeeper instance, and create the Timestamp Table and the Commit Table using the omid.sh script included in the bin directory of the tso server:</p>
-<div>
-<div>
-<pre class="source">$ bin/omid.sh create-hbase-commit-table -numRegions 16
+<div class="source">
+<div class="source"><pre class="prettyprint">$ bin/omid.sh create-hbase-commit-table -numRegions 16
$ bin/omid.sh create-hbase-timestamp-table
</pre></div></div>
-
<p>These two tables are required by Omid and they must not be accessed by client applications.</p></div>
<div class="section">
<h3><a name="a5._Start_the_TSO_Server"></a>5. Start the TSO Server</h3>
-<div>
-<div>
-<pre class="source">$ bin/omid.sh tso
+<div class="source">
+<div class="source"><pre class="prettyprint">$ bin/omid.sh tso
</pre></div></div>
-
<p>This starts the TSO server that in turn will connect to HBase to store information in HBase. By default the TSO listens on port 54758. If you want to change the TSO configuration, you can modify the contents in the conf/omid.conf file.</p></div></div>
<div class="section">
<h2><a name="HBase_Client_Usage"></a>HBase Client Usage</h2>
@@ -277,36 +285,31 @@ $ bin/omid.sh create-hbase-timestamp-table
<h3><a name="a2._Add_hbase-client_Dependency"></a>2. Add hbase-client Dependency</h3>
<p>Choose the right version of the hbase-client jar. For example, in a Maven-based app add the following dependency in the pom.xml file:</p>
-<div>
-<div>
-<pre class="source"><dependency>
+<div class="source">
+<div class="source"><pre class="prettyprint"><dependency>
<groupId>com.yahoo.omid</groupId>
<artifactId>hbase-client</artifactId>
<version>${hbase_client.version}</version>
</dependency>
-</pre></div></div>
-</div>
+</pre></div></div></div>
<div class="section">
<h3><a name="a3._Start_Coding_Using_the_Omid_Client_Interfaces"></a>3. Start Coding Using the Omid Client Interfaces</h3>
<p>In Omid there are two client interfaces: <tt>TTable</tt> and <tt>TransactionManager</tt> (<i>These interfaces will likely change slightly in future.</i>):</p>
-<ol style="list-style-type: decimal">
+<ol style="list-style-type: decimal">
+
<li>
-
-<p>The <tt>TransactionManager</tt> is used for creating transactional contexts, that is, transactions. A builder is provided in the <tt>HBaseTransactionManager</tt> class in order to get the TransactionManager interface.</p>
-</li>
+<p>The <tt>TransactionManager</tt> is used for creating transactional contexts, that is, transactions. A builder is provided in the <tt>HBaseTransactionManager</tt> class in order to get the TransactionManager interface.</p></li>
+
<li>
-
-<p><tt>TTable</tt> is used for putting, getting and scanning entries in a HBase table. TTable’s interface is similar to the standard <tt>HTableInterface</tt>, and only requires passing the transactional context as a first parameter in the transactional aware methods (e.g. <tt>put(Transaction tx, Put put)</tt>)</p>
-</li>
+<p><tt>TTable</tt> is used for putting, getting and scanning entries in a HBase table. TTable’s interface is similar to the standard <tt>HTableInterface</tt>, and only requires passing the transactional context as a first parameter in the transactional aware methods (e.g. <tt>put(Transaction tx, Put put)</tt>)</p></li>
</ol></div></div>
<div class="section">
<h2><a name="Example_Application"></a>Example Application</h2>
<p>Below is provided a sample application accessing data transactionally. Its a dummy application that writes two cells in two different rows of a table in a transactional context, but is enough to show how the different Omid client APIs are used. A detailed explanation of the client interfaces can be found in the <a href="basic-examples.html">Basic Examples</a> section.</p>
-<div>
-<div>
-<pre class="source">import org.apache.hadoop.hbase.client.Put;
+<div class="source">
+<div class="source"><pre class="prettyprint">import org.apache.hadoop.hbase.client.Put;
import org.apache.hadoop.hbase.util.Bytes;
import com.yahoo.omid.transaction.HBaseTransactionManager;
import com.yahoo.omid.transaction.TTable;
@@ -341,12 +344,10 @@ public class OmidExample {
}
</pre></div></div>
-
<p>To run the application, make sure <tt>core-site.xml</tt> and <tt>hbase-site.xml</tt> for your HBase cluster are present in your CLASSPATH. The default configuration settings for the Omid client are loaded from the <tt>default-hbase-omid-client-config.yml</tt> file. In order to change the client default settings, you can either; 1) put your specific configuration settings in a file named <tt>hbase-omid-client-config.yml</tt> and include it in the CLASSPATH; or 2) do it programmatically in the application code by creating an instance of the <tt>HBaseOmidClientConfiguration</tt> class and passing it in the creation of the transaction manager:</p>
-<div>
-<div>
-<pre class="source"> import com.yahoo.omid.transaction.HBaseOmidClientConfiguration;
+<div class="source">
+<div class="source"><pre class="prettyprint"> import com.yahoo.omid.transaction.HBaseOmidClientConfiguration;
...
@@ -360,51 +361,44 @@ public class OmidExample {
...
</pre></div></div>
-
<p>Also, you will need to create a HBase table “MY_TX_TABLE”, with column family “MY_CF”, and with <tt>TTL</tt> disabled and <tt>VERSIONS</tt> set to <tt>Integer.MAX_VALUE</tt>. For example using the HBase shell:</p>
-<div>
-<div>
-<pre class="source">create 'MY_TX_TABLE', {NAME => 'MY_CF', VERSIONS => '2147483647', TTL => '2147483647'}
+<div class="source">
+<div class="source"><pre class="prettyprint">create 'MY_TX_TABLE', {NAME => 'MY_CF', VERSIONS => '2147483647', TTL => '2147483647'}
</pre></div></div>
-
<p>This example assumes non-secure communication with HBase. If your HBase cluster is secured with Kerberos, you will need to use the <tt>UserGroupInformation</tt> API to log in securely.</p></div>
<div class="section">
<h2><a name="The_Omid_Compactor_Coprocessor"></a>The Omid Compactor Coprocessor</h2>
<p>Omid includes a jar with an HBase coprocessor for performing data cleanup that operates during compactions, both minor and major. Specifically, it does the following:</p>
-<ul>
+<ul>
+
<li>Cleans up garbage data from aborted transactions</li>
-<li>Purges deleted cells. Omid deletes work by placing a special tombstone marker in cells. The compactor detects these and actually purges data when it is safe to do so (i.e. when there are no committable transactions that may read the data).</li>
+
+<li>Purges deleted cells. Omid deletes work by placing a special tombstone marker in cells. The compactor detects these and actually purges data when it is safe to do so (i.e. when there are no committable transactions that may read the data).</li>
+
<li>‘Heals’ committed cells for which the writer failed to write shadow cells.</li>
</ul>
<p>To deploy the coprocessor, the coprocessor jar must be placed in a location (typically on HDFS) that is accessible by HBase region servers. The coprocessor may then be enabled on a transactional table by the following steps in the HBase shell:</p>
<p><b>1) Disable the table</b></p>
-<div>
-<div>
-<pre class="source">disable 'MY_TX_TABLE'
+<div class="source">
+<div class="source"><pre class="prettyprint">disable 'MY_TX_TABLE'
</pre></div></div>
-
<p><b>2) Add a coprocessor specification to the table via a “coprocessor” attribute. The coprocessor spec may (and usually will) also include the name of the Omid commit table</b></p>
-<div>
-<div>
-<pre class="source">alter 'MY_TX_TABLE', METHOD => 'table_att', 'coprocessor'=>'<path_to_omid_coprocessor>/omid-hbase-coprocessor-<coprocessor_version>.jar|com.yahoo.omid.transaction.OmidCompactor|1001|omid.committable.tablename=OMID_COMMIT_TABLE'
+<div class="source">
+<div class="source"><pre class="prettyprint">alter 'MY_TX_TABLE', METHOD => 'table_att', 'coprocessor'=>'<path_to_omid_coprocessor>/omid-hbase-coprocessor-<coprocessor_version>.jar|com.yahoo.omid.transaction.OmidCompactor|1001|omid.committable.tablename=OMID_COMMIT_TABLE'
</pre></div></div>
-
<p><b>3) Add an “OMID_ENABLED => true” flag to any column families which the co-processor should work on</b></p>
-<div>
-<div>
-<pre class="source">alter 'MY_TX_TABLE', { NAME => 'MY_CF', METADATA => {'OMID_ENABLED' => 'true'}}
+<div class="source">
+<div class="source"><pre class="prettyprint">alter 'MY_TX_TABLE', { NAME => 'MY_CF', METADATA => {'OMID_ENABLED' => 'true'}}
</pre></div></div>
-
<p><b>4) Re-enable the table</b></p>
-<div>
-<div>
-<pre class="source">enable 'MY_TX_TABLE'
+<div class="source">
+<div class="source"><pre class="prettyprint">enable 'MY_TX_TABLE'
</pre></div></div></div>
</div>
</div>
http://git-wip-us.apache.org/repos/asf/incubator-omid-site/blob/ff061775/source-repository.html
----------------------------------------------------------------------
diff --git a/source-repository.html b/source-repository.html
index 3f4e529..5384c9b 100644
--- a/source-repository.html
+++ b/source-repository.html
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia Site Renderer 1.7 at 2018-09-20
+ | Generated by Apache Maven Doxia Site Renderer 1.7 at 2018-09-26
| Rendered using Apache Maven Fluido Skin 1.5
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20180920" />
+ <meta name="Date-Revision-yyyymmdd" content="20180926" />
<meta http-equiv="Content-Language" content="en" />
<title>Apache Omid – Source Code Management</title>
<link rel="stylesheet" href="./css/apache-maven-fluido-1.5.min.css" />
@@ -198,7 +198,7 @@
</li>
- <li id="publishDate" class="pull-right">Last Published: 2018-09-20</li>
+ <li id="publishDate" class="pull-right">Last Published: 2018-09-26</li>
</ul>
</div>
http://git-wip-us.apache.org/repos/asf/incubator-omid-site/blob/ff061775/team-list.html
----------------------------------------------------------------------
diff --git a/team-list.html b/team-list.html
index bf0047c..428fe46 100644
--- a/team-list.html
+++ b/team-list.html
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia Site Renderer 1.7 at 2018-09-20
+ | Generated by Apache Maven Doxia Site Renderer 1.7 at 2018-09-26
| Rendered using Apache Maven Fluido Skin 1.5
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20180920" />
+ <meta name="Date-Revision-yyyymmdd" content="20180926" />
<meta http-equiv="Content-Language" content="en" />
<title>Apache Omid – Project Team</title>
<link rel="stylesheet" href="./css/apache-maven-fluido-1.5.min.css" />
@@ -198,7 +198,7 @@
</li>
- <li id="publishDate" class="pull-right">Last Published: 2018-09-20</li>
+ <li id="publishDate" class="pull-right">Last Published: 2018-09-26</li>
</ul>
</div>
http://git-wip-us.apache.org/repos/asf/incubator-omid-site/blob/ff061775/xref-test/allclasses-frame.html
----------------------------------------------------------------------
diff --git a/xref-test/allclasses-frame.html b/xref-test/allclasses-frame.html
index 608b7da..03736aa 100644
--- a/xref-test/allclasses-frame.html
+++ b/xref-test/allclasses-frame.html
@@ -69,9 +69,6 @@
<a href="org/apache/omid/transaction/TSOForHBaseCompactorTestModule.html" target="classFrame">TSOForHBaseCompactorTestModule</a>
</li>
<li>
- <a href="org/apache/omid/transaction/TSOForSnapshotFilterTestModule.html" target="classFrame">TSOForSnapshotFilterTestModule</a>
- </li>
- <li>
<a href="org/apache/omid/tso/TSOMockModule.html" target="classFrame">TSOMockModule</a>
</li>
<li>
@@ -93,10 +90,13 @@
<a href="org/apache/omid/tso/TestBatch.html" target="classFrame">TestBatch</a>
</li>
<li>
+ <a href="org/apache/omid/tso/TestBatchPool.html" target="classFrame">TestBatchPool</a>
+ </li>
+ <li>
<a href="org/apache/omid/transaction/TestCellUtils.html" target="classFrame">TestCellUtils</a>
</li>
<li>
- <a href="org/apache/omid/transaction/TestCheckpoint.html" target="classFrame">TestCheckpoint</a>
+ <a href="org/apache/omid/TestReflectionHelper.html" target="classFrame">TestClass</a>
</li>
<li>
<a href="org/apache/omid/transaction/TestColumnIterator.html" target="classFrame">TestColumnIterator</a>
@@ -150,9 +150,6 @@
<a href="org/apache/omid/tso/TestLongCache.html" target="classFrame">TestLongCache</a>
</li>
<li>
- <a href="org/apache/omid/transaction/TestMarkPutAsCommitted.html" target="classFrame">TestMarkPutAsCommitted</a>
- </li>
- <li>
<a href="org/apache/omid/tso/client/TestMockTSOClient.html" target="classFrame">TestMockTSOClient</a>
</li>
<li>
@@ -177,6 +174,9 @@
<a href="org/apache/omid/transaction/TestReadPath.html" target="classFrame">TestReadPath</a>
</li>
<li>
+ <a href="org/apache/omid/TestReflectionHelper.html" target="classFrame">TestReflectionHelper</a>
+ </li>
+ <li>
<a href="org/apache/omid/tso/TestReplyProcessor.html" target="classFrame">TestReplyProcessor</a>
</li>
<li>
@@ -192,9 +192,6 @@
<a href="org/apache/omid/transaction/TestSingleColumnFamily.html" target="classFrame">TestSingleColumnFamily</a>
</li>
<li>
- <a href="org/apache/omid/transaction/TestSnapshotFilter.html" target="classFrame">TestSnapshotFilter</a>
- </li>
- <li>
<a href="org/apache/statemachine/TestStateMachine.html" target="classFrame">TestStateMachine</a>
</li>
<li>
@@ -210,9 +207,6 @@
<a href="org/apache/omid/tso/client/TestTSOClientResponseHandling.html" target="classFrame">TestTSOClientResponseHandling</a>
</li>
<li>
- <a href="org/apache/omid/tso/client/TestTSOClientRowAndCellLevelConflict.html" target="classFrame">TestTSOClientRowAndCellLevelConflict</a>
- </li>
- <li>
<a href="org/apache/omid/transaction/TestTSOModule.html" target="classFrame">TestTSOModule</a>
</li>
<li>
@@ -243,9 +237,6 @@
<a href="org/apache/omid/TestUtils.html" target="classFrame">TestUtils</a>
</li>
<li>
- <a href="org/apache/omid/tso/TestWorldTimeOracle.html" target="classFrame">TestWorldTimeOracle</a>
- </li>
- <li>
<a href="org/apache/omid/timestamp/storage/TestZKTimestampStorage.html" target="classFrame">TestZKTimestampStorage</a>
</li>
<li>
http://git-wip-us.apache.org/repos/asf/incubator-omid-site/blob/ff061775/xref-test/index.html
----------------------------------------------------------------------
diff --git a/xref-test/index.html b/xref-test/index.html
index 2834caf..3d49b33 100644
--- a/xref-test/index.html
+++ b/xref-test/index.html
@@ -3,7 +3,7 @@
<html xml:lang="en" lang="en">
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8" />
- <title>Omid 0.8.2.11-SNAPSHOT Reference</title>
+ <title>Omid 0.9.0.0 Reference</title>
</head>
<frameset cols="20%,80%">
<frameset rows="30%,70%">