You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kudu.apache.org by ab...@apache.org on 2018/10/26 19:04:20 UTC
[50/51] [partial] kudu-site git commit: Publish commit(s) from site
source repo: a05466438 [blog] Add post about 1.8.0 release 1fefa84c7 Updating
web site for Kudu 1.8.0 release 637a50027 [site] Add http to https redirect
40f26d899 gh-pages: Make
http://git-wip-us.apache.org/repos/asf/kudu-site/blob/854be1d3/.htaccess
----------------------------------------------------------------------
diff --git a/.htaccess b/.htaccess
index 4ace0af..ce7dc59 100644
--- a/.htaccess
+++ b/.htaccess
@@ -25,4 +25,8 @@
# Server should support REQUEST_SCHEME and should be running http or https.
RewriteCond "%{REQUEST_SCHEME}" "^http"
RewriteRule ^/?(.*)$ %{REQUEST_SCHEME}://kudu.apache.org/$1 [R=301,L]
+
+ # Redirect http to https
+ RewriteCond %{HTTPS} off
+ RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI} [R=301,L]
</IfModule>
http://git-wip-us.apache.org/repos/asf/kudu-site/blob/854be1d3/2016/02/26/apache-kudu-0-7-0-released.html
----------------------------------------------------------------------
diff --git a/2016/02/26/apache-kudu-0-7-0-released.html b/2016/02/26/apache-kudu-0-7-0-released.html
index 1f1d7b5..b068f40 100644
--- a/2016/02/26/apache-kudu-0-7-0-released.html
+++ b/2016/02/26/apache-kudu-0-7-0-released.html
@@ -126,7 +126,7 @@ part of the ASF Incubator, version 0.7.0!</p>
<!--more-->
<ul>
- <li>Kudu now has limited support for Apache Spark, through a simple <code class="highlighter-rouge">kuduRDD</code> API</li>
+ <li>Kudu now has limited support for Apache Spark, through a simple <code>kuduRDD</code> API</li>
<li>Several improvements to the Java client including better error handling support and fixes for a potential client hang.</li>
<li>Better platform support, including support for building on RHEL 7, CentOS 7, and SLES 12.</li>
<li>The Kudu Python client has been completely revamped and is now considered a supported feature.</li>
@@ -149,6 +149,8 @@ part of the ASF Incubator, version 0.7.0!</p>
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2018/10/26/apache-kudu-1-8-0-released.html">Apache Kudu 1.8.0 Released</a> </li>
+
<li> <a href="/2018/09/26/index-skip-scan-optimization-in-kudu.html">Index Skip Scan Optimization in Kudu</a> </li>
<li> <a href="/2018/09/11/simplified-pipelines-with-kudu.html">Simplified Data Pipelines with Kudu</a> </li>
@@ -177,8 +179,6 @@ part of the ASF Incubator, version 0.7.0!</p>
<li> <a href="/2016/11/15/weekly-update.html">Apache Kudu Weekly Update November 15th, 2016</a> </li>
- <li> <a href="/2016/11/01/weekly-update.html">Apache Kudu Weekly Update November 1st, 2016</a> </li>
-
</ul>
</div>
</div>
http://git-wip-us.apache.org/repos/asf/kudu-site/blob/854be1d3/2016/03/10/apache-kudu-0-7-1-released.html
----------------------------------------------------------------------
diff --git a/2016/03/10/apache-kudu-0-7-1-released.html b/2016/03/10/apache-kudu-0-7-1-released.html
index f1eeed8..33e5350 100644
--- a/2016/03/10/apache-kudu-0-7-1-released.html
+++ b/2016/03/10/apache-kudu-0-7-1-released.html
@@ -148,6 +148,8 @@ bugs fixed in this release:
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2018/10/26/apache-kudu-1-8-0-released.html">Apache Kudu 1.8.0 Released</a> </li>
+
<li> <a href="/2018/09/26/index-skip-scan-optimization-in-kudu.html">Index Skip Scan Optimization in Kudu</a> </li>
<li> <a href="/2018/09/11/simplified-pipelines-with-kudu.html">Simplified Data Pipelines with Kudu</a> </li>
@@ -176,8 +178,6 @@ bugs fixed in this release:
<li> <a href="/2016/11/15/weekly-update.html">Apache Kudu Weekly Update November 15th, 2016</a> </li>
- <li> <a href="/2016/11/01/weekly-update.html">Apache Kudu Weekly Update November 1st, 2016</a> </li>
-
</ul>
</div>
</div>
http://git-wip-us.apache.org/repos/asf/kudu-site/blob/854be1d3/2016/03/20/weekly-update.html
----------------------------------------------------------------------
diff --git a/2016/03/20/weekly-update.html b/2016/03/20/weekly-update.html
index 2e00f02..107e9b7 100644
--- a/2016/03/20/weekly-update.html
+++ b/2016/03/20/weekly-update.html
@@ -208,6 +208,8 @@ at the <a href="http://www.meetup.com/Rust-Bay-Area/events/229107276/">Rust Bay
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2018/10/26/apache-kudu-1-8-0-released.html">Apache Kudu 1.8.0 Released</a> </li>
+
<li> <a href="/2018/09/26/index-skip-scan-optimization-in-kudu.html">Index Skip Scan Optimization in Kudu</a> </li>
<li> <a href="/2018/09/11/simplified-pipelines-with-kudu.html">Simplified Data Pipelines with Kudu</a> </li>
@@ -236,8 +238,6 @@ at the <a href="http://www.meetup.com/Rust-Bay-Area/events/229107276/">Rust Bay
<li> <a href="/2016/11/15/weekly-update.html">Apache Kudu Weekly Update November 15th, 2016</a> </li>
- <li> <a href="/2016/11/01/weekly-update.html">Apache Kudu Weekly Update November 1st, 2016</a> </li>
-
</ul>
</div>
</div>
http://git-wip-us.apache.org/repos/asf/kudu-site/blob/854be1d3/2016/03/28/weekly-update.html
----------------------------------------------------------------------
diff --git a/2016/03/28/weekly-update.html b/2016/03/28/weekly-update.html
index 54e625c..c94af87 100644
--- a/2016/03/28/weekly-update.html
+++ b/2016/03/28/weekly-update.html
@@ -199,6 +199,8 @@ in San Jose. The conference will feature two talks on Kudu:</li>
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2018/10/26/apache-kudu-1-8-0-released.html">Apache Kudu 1.8.0 Released</a> </li>
+
<li> <a href="/2018/09/26/index-skip-scan-optimization-in-kudu.html">Index Skip Scan Optimization in Kudu</a> </li>
<li> <a href="/2018/09/11/simplified-pipelines-with-kudu.html">Simplified Data Pipelines with Kudu</a> </li>
@@ -227,8 +229,6 @@ in San Jose. The conference will feature two talks on Kudu:</li>
<li> <a href="/2016/11/15/weekly-update.html">Apache Kudu Weekly Update November 15th, 2016</a> </li>
- <li> <a href="/2016/11/01/weekly-update.html">Apache Kudu Weekly Update November 1st, 2016</a> </li>
-
</ul>
</div>
</div>
http://git-wip-us.apache.org/repos/asf/kudu-site/blob/854be1d3/2016/04/04/weekly-update.html
----------------------------------------------------------------------
diff --git a/2016/04/04/weekly-update.html b/2016/04/04/weekly-update.html
index ff1ec3c..f53714f 100644
--- a/2016/04/04/weekly-update.html
+++ b/2016/04/04/weekly-update.html
@@ -218,6 +218,8 @@ on Friday, April 8th.</li>
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2018/10/26/apache-kudu-1-8-0-released.html">Apache Kudu 1.8.0 Released</a> </li>
+
<li> <a href="/2018/09/26/index-skip-scan-optimization-in-kudu.html">Index Skip Scan Optimization in Kudu</a> </li>
<li> <a href="/2018/09/11/simplified-pipelines-with-kudu.html">Simplified Data Pipelines with Kudu</a> </li>
@@ -246,8 +248,6 @@ on Friday, April 8th.</li>
<li> <a href="/2016/11/15/weekly-update.html">Apache Kudu Weekly Update November 15th, 2016</a> </li>
- <li> <a href="/2016/11/01/weekly-update.html">Apache Kudu Weekly Update November 1st, 2016</a> </li>
-
</ul>
</div>
</div>
http://git-wip-us.apache.org/repos/asf/kudu-site/blob/854be1d3/2016/04/11/weekly-update.html
----------------------------------------------------------------------
diff --git a/2016/04/11/weekly-update.html b/2016/04/11/weekly-update.html
index 663ca93..2266586 100644
--- a/2016/04/11/weekly-update.html
+++ b/2016/04/11/weekly-update.html
@@ -218,6 +218,8 @@ please pipe up on the mailing list.</p>
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2018/10/26/apache-kudu-1-8-0-released.html">Apache Kudu 1.8.0 Released</a> </li>
+
<li> <a href="/2018/09/26/index-skip-scan-optimization-in-kudu.html">Index Skip Scan Optimization in Kudu</a> </li>
<li> <a href="/2018/09/11/simplified-pipelines-with-kudu.html">Simplified Data Pipelines with Kudu</a> </li>
@@ -246,8 +248,6 @@ please pipe up on the mailing list.</p>
<li> <a href="/2016/11/15/weekly-update.html">Apache Kudu Weekly Update November 15th, 2016</a> </li>
- <li> <a href="/2016/11/01/weekly-update.html">Apache Kudu Weekly Update November 1st, 2016</a> </li>
-
</ul>
</div>
</div>
http://git-wip-us.apache.org/repos/asf/kudu-site/blob/854be1d3/2016/04/12/apache-kudu-0-8-0-released.html
----------------------------------------------------------------------
diff --git a/2016/04/12/apache-kudu-0-8-0-released.html b/2016/04/12/apache-kudu-0-8-0-released.html
index f5254c2..0b133da 100644
--- a/2016/04/12/apache-kudu-0-8-0-released.html
+++ b/2016/04/12/apache-kudu-0-8-0-released.html
@@ -139,6 +139,8 @@ client, plus many other improvements and bug fixes.</p>
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2018/10/26/apache-kudu-1-8-0-released.html">Apache Kudu 1.8.0 Released</a> </li>
+
<li> <a href="/2018/09/26/index-skip-scan-optimization-in-kudu.html">Index Skip Scan Optimization in Kudu</a> </li>
<li> <a href="/2018/09/11/simplified-pipelines-with-kudu.html">Simplified Data Pipelines with Kudu</a> </li>
@@ -167,8 +169,6 @@ client, plus many other improvements and bug fixes.</p>
<li> <a href="/2016/11/15/weekly-update.html">Apache Kudu Weekly Update November 15th, 2016</a> </li>
- <li> <a href="/2016/11/01/weekly-update.html">Apache Kudu Weekly Update November 1st, 2016</a> </li>
-
</ul>
</div>
</div>
http://git-wip-us.apache.org/repos/asf/kudu-site/blob/854be1d3/2016/04/14/ingesting-json-apache-kudu-streamsets-data-collector.html
----------------------------------------------------------------------
diff --git a/2016/04/14/ingesting-json-apache-kudu-streamsets-data-collector.html b/2016/04/14/ingesting-json-apache-kudu-streamsets-data-collector.html
index 447da2e..ab0c442 100644
--- a/2016/04/14/ingesting-json-apache-kudu-streamsets-data-collector.html
+++ b/2016/04/14/ingesting-json-apache-kudu-streamsets-data-collector.html
@@ -162,6 +162,8 @@ kernel drivers in C to web front ends in JavaScript.</p>
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2018/10/26/apache-kudu-1-8-0-released.html">Apache Kudu 1.8.0 Released</a> </li>
+
<li> <a href="/2018/09/26/index-skip-scan-optimization-in-kudu.html">Index Skip Scan Optimization in Kudu</a> </li>
<li> <a href="/2018/09/11/simplified-pipelines-with-kudu.html">Simplified Data Pipelines with Kudu</a> </li>
@@ -190,8 +192,6 @@ kernel drivers in C to web front ends in JavaScript.</p>
<li> <a href="/2016/11/15/weekly-update.html">Apache Kudu Weekly Update November 15th, 2016</a> </li>
- <li> <a href="/2016/11/01/weekly-update.html">Apache Kudu Weekly Update November 1st, 2016</a> </li>
-
</ul>
</div>
</div>
http://git-wip-us.apache.org/repos/asf/kudu-site/blob/854be1d3/2016/04/18/weekly-update.html
----------------------------------------------------------------------
diff --git a/2016/04/18/weekly-update.html b/2016/04/18/weekly-update.html
index 17607de..ed8e02a 100644
--- a/2016/04/18/weekly-update.html
+++ b/2016/04/18/weekly-update.html
@@ -216,6 +216,8 @@ Data Collector</a>.</li>
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2018/10/26/apache-kudu-1-8-0-released.html">Apache Kudu 1.8.0 Released</a> </li>
+
<li> <a href="/2018/09/26/index-skip-scan-optimization-in-kudu.html">Index Skip Scan Optimization in Kudu</a> </li>
<li> <a href="/2018/09/11/simplified-pipelines-with-kudu.html">Simplified Data Pipelines with Kudu</a> </li>
@@ -244,8 +246,6 @@ Data Collector</a>.</li>
<li> <a href="/2016/11/15/weekly-update.html">Apache Kudu Weekly Update November 15th, 2016</a> </li>
- <li> <a href="/2016/11/01/weekly-update.html">Apache Kudu Weekly Update November 1st, 2016</a> </li>
-
</ul>
</div>
</div>
http://git-wip-us.apache.org/repos/asf/kudu-site/blob/854be1d3/2016/04/19/kudu-0-8-0-predicate-improvements.html
----------------------------------------------------------------------
diff --git a/2016/04/19/kudu-0-8-0-predicate-improvements.html b/2016/04/19/kudu-0-8-0-predicate-improvements.html
index aa0742b..0b5065b 100644
--- a/2016/04/19/kudu-0-8-0-predicate-improvements.html
+++ b/2016/04/19/kudu-0-8-0-predicate-improvements.html
@@ -157,10 +157,10 @@ know on the <a href="https://issues.apache.org/jira/browse/KUDU-1065">JIRA issue
<h2 id="redesigned-predicate-api-in-the-java-client">Redesigned Predicate API in the Java Client</h2>
<p>The Java client has a new way to express scan predicates: the
-<a href="http://kudu.apache.org/apidocs/org/kududb/client/KuduPredicate.html"><code class="highlighter-rouge">KuduPredicate</code></a>.
+<a href="http://kudu.apache.org/apidocs/org/kududb/client/KuduPredicate.html"><code>KuduPredicate</code></a>.
The API matches the corresponding C++ API more closely, and adds support for
specifying exclusive, as well as inclusive, range predicates. The existing
-<a href="http://kudu.apache.org/apidocs/org/kududb/client/ColumnRangePredicate.html"><code class="highlighter-rouge">ColumnRangePredicate</code></a>
+<a href="http://kudu.apache.org/apidocs/org/kududb/client/ColumnRangePredicate.html"><code>ColumnRangePredicate</code></a>
API has been deprecated, and will be removed soon. Example of transitioning from
the old to new API:</p>
@@ -178,14 +178,14 @@ the old to new API:</p>
<h2 id="under-the-covers-changes">Under the Covers Changes</h2>
-<p>The scan optimizations in the server and C++ client, and the new <code class="highlighter-rouge">KuduPredicate</code>
+<p>The scan optimizations in the server and C++ client, and the new <code>KuduPredicate</code>
API in the Java client are made possible by an overhaul of how predicates are
handled internally. A new protobuf message type,
-<a href="https://github.com/apache/incubator-kudu/blob/master/src/kudu/common/common.proto#L273"><code class="highlighter-rouge">ColumnPredicatePB</code></a>
+<a href="https://github.com/apache/incubator-kudu/blob/master/src/kudu/common/common.proto#L273"><code>ColumnPredicatePB</code></a>
has been introduced, and will allow more column predicate types to be introduced
in the future. If you are interested in contributing to Kudu but don’t know
-where to start, consider adding a new predicate type; for example the <code class="highlighter-rouge">IS NULL</code>,
-<code class="highlighter-rouge">IS NOT NULL</code>, <code class="highlighter-rouge">IN</code>, and <code class="highlighter-rouge">LIKE</code> predicates types are currently not implemented.</p>
+where to start, consider adding a new predicate type; for example the <code>IS NULL</code>,
+<code>IS NOT NULL</code>, <code>IN</code>, and <code>LIKE</code> predicates types are currently not implemented.</p>
</div>
</article>
@@ -196,6 +196,8 @@ where to start, consider adding a new predicate type; for example the <code clas
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2018/10/26/apache-kudu-1-8-0-released.html">Apache Kudu 1.8.0 Released</a> </li>
+
<li> <a href="/2018/09/26/index-skip-scan-optimization-in-kudu.html">Index Skip Scan Optimization in Kudu</a> </li>
<li> <a href="/2018/09/11/simplified-pipelines-with-kudu.html">Simplified Data Pipelines with Kudu</a> </li>
@@ -224,8 +226,6 @@ where to start, consider adding a new predicate type; for example the <code clas
<li> <a href="/2016/11/15/weekly-update.html">Apache Kudu Weekly Update November 15th, 2016</a> </li>
- <li> <a href="/2016/11/01/weekly-update.html">Apache Kudu Weekly Update November 1st, 2016</a> </li>
-
</ul>
</div>
</div>
http://git-wip-us.apache.org/repos/asf/kudu-site/blob/854be1d3/2016/04/25/weekly-update.html
----------------------------------------------------------------------
diff --git a/2016/04/25/weekly-update.html b/2016/04/25/weekly-update.html
index fa75dea..bfc2219 100644
--- a/2016/04/25/weekly-update.html
+++ b/2016/04/25/weekly-update.html
@@ -215,6 +215,8 @@ Kudu 0.8</a>.</li>
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2018/10/26/apache-kudu-1-8-0-released.html">Apache Kudu 1.8.0 Released</a> </li>
+
<li> <a href="/2018/09/26/index-skip-scan-optimization-in-kudu.html">Index Skip Scan Optimization in Kudu</a> </li>
<li> <a href="/2018/09/11/simplified-pipelines-with-kudu.html">Simplified Data Pipelines with Kudu</a> </li>
@@ -243,8 +245,6 @@ Kudu 0.8</a>.</li>
<li> <a href="/2016/11/15/weekly-update.html">Apache Kudu Weekly Update November 15th, 2016</a> </li>
- <li> <a href="/2016/11/01/weekly-update.html">Apache Kudu Weekly Update November 1st, 2016</a> </li>
-
</ul>
</div>
</div>
http://git-wip-us.apache.org/repos/asf/kudu-site/blob/854be1d3/2016/04/26/ycsb.html
----------------------------------------------------------------------
diff --git a/2016/04/26/ycsb.html b/2016/04/26/ycsb.html
index 6d549d7..1276904 100644
--- a/2016/04/26/ycsb.html
+++ b/2016/04/26/ycsb.html
@@ -145,12 +145,12 @@
</ul>
<h2 id="experimental-setup">Experimental setup</h2>
-<p>The single-node Kudu cluster was configured, started, and stopped by a Python script <code class="highlighter-rouge">run_experiments.py</code> which cycled through several different configurations, completely removing all data in between each iteration. For each Kudu configuration, YCSB was used to load 100M rows of data (each approximately 1KB). YCSB is configured with 16 client threads on the same node. For each configuration, the YCSB log as well as periodic dumps of Tablet Server metrics are captured for later analysis.</p>
+<p>The single-node Kudu cluster was configured, started, and stopped by a Python script <code>run_experiments.py</code> which cycled through several different configurations, completely removing all data in between each iteration. For each Kudu configuration, YCSB was used to load 100M rows of data (each approximately 1KB). YCSB is configured with 16 client threads on the same node. For each configuration, the YCSB log as well as periodic dumps of Tablet Server metrics are captured for later analysis.</p>
<p>Note that in many cases, the 16 client threads were not enough to max out the full performance of the machine. These experiments should not be taken to determine the maximum throughput of Kudu – instead, we are looking at comparing the <em>relative</em> performance of different configuration options.</p>
<h1 id="benchmarking-synchronous-insert-operations">Benchmarking Synchronous Insert Operations</h1>
-<p>The first set of experiments runs the YCSB load with the <code class="highlighter-rouge">sync_ops=true</code> configuration option. This option means that each client thread will insert one row at a time and synchronously wait for the response before inserting the next row. The lack of batching makes this a good stress test for Kudu’s RPC performance and other fixed per-request costs.</p>
+<p>The first set of experiments runs the YCSB load with the <code>sync_ops=true</code> configuration option. This option means that each client thread will insert one row at a time and synchronously wait for the response before inserting the next row. The lack of batching makes this a good stress test for Kudu’s RPC performance and other fixed per-request costs.</p>
<p>The fact that the requests are synchronous also makes it easy to measure the <em>latency</em> of the write requests. With request batching enabled, latency would be irrelevant.</p>
@@ -170,25 +170,25 @@
<p><img src="/img/YCSB_files/YCSB_3_0.png" alt="png" class="img-responsive" /></p>
-<div class="highlighter-rouge">Average throughput: 31163 ops/sec
-</div>
+<pre><code>Average throughput: 31163 ops/sec
+</code></pre>
<p>The results here are interesting: the throughput starts out around 70K rows/second, but then collapses to nearly zero. After staying near zero for a while, it shoots back up to the original performance, and the pattern repeats many times.</p>
<p>Also note that the 99th percentile latency seems to alternate between close to zero and a value near 500ms. This bimodal distribution led me to grep in the Java source for the magic number 500. Sure enough, I found:</p>
-<div class="highlighter-rouge"><span class="kd">public</span> <span class="kd">static</span> <span class="kd">final</span> <span class="kt">int</span> <span class="n">SLEEP_TIME</span> <span class="o">=</span> <span class="mi">500</span><span class="o">;</span>
-</div>
+<pre><code class="language-java">public static final int SLEEP_TIME = 500;
+</code></pre>
<p>Used in this backoff calculation method (slightly paraphrased here):</p>
-<div class="highlighter-rouge"> <span class="kt">long</span> <span class="nf">getSleepTimeForRpc</span><span class="o">(</span><span class="n">KuduRpc</span><span class="o"><?></span> <span class="n">rpc</span><span class="o">)</span> <span class="o">{</span>
- <span class="c1">// TODO backoffs? Sleep in increments of 500 ms, plus some random time up to 50</span>
- <span class="k">return</span> <span class="o">(</span><span class="n">attemptCount</span> <span class="o">*</span> <span class="n">SLEEP_TIME</span><span class="o">)</span> <span class="o">+</span> <span class="n">sleepRandomizer</span><span class="o">.</span><span class="na">nextInt</span><span class="o">(</span><span class="mi">50</span><span class="o">);</span>
- <span class="o">}</span>
-</div>
+<pre><code class="language-java"> long getSleepTimeForRpc(KuduRpc<?> rpc) {
+ // TODO backoffs? Sleep in increments of 500 ms, plus some random time up to 50
+ return (attemptCount * SLEEP_TIME) + sleepRandomizer.nextInt(50);
+ }
+</code></pre>
-<p>One reason that a client will back off and retry is a <code class="highlighter-rouge">SERVER_TOO_BUSY</code> response from the server. This response is used in a number of overload situations. In a write-mostly workload, the most likely situation is that the server is low on memory and thus asking clients to back off while it flushes. Sure enough, when we graph the heap usage over time, as well as the rate of writes rejected due to low-memory, we see that this is the case:</p>
+<p>One reason that a client will back off and retry is a <code>SERVER_TOO_BUSY</code> response from the server. This response is used in a number of overload situations. In a write-mostly workload, the most likely situation is that the server is low on memory and thus asking clients to back off while it flushes. Sure enough, when we graph the heap usage over time, as well as the rate of writes rejected due to low-memory, we see that this is the case:</p>
<div class="highlight"><pre><code class="language-python" data-lang="python"><span class="n">plot_ts_metric</span><span class="p">(</span><span class="n">data</span><span class="p">[</span><span class="s">'default'</span><span class="p">],</span> <span class="s">"heap_allocated"</span><span class="p">,</span> <span class="s">"Heap usage (GB)"</span><span class="p">,</span> <span class="mi">1024</span><span class="o">*</span><span class="mi">1024</span><span class="o">*</span><span class="mi">1024</span><span class="p">)</span>
<span class="n">plot_ts_metric</span><span class="p">(</span><span class="n">data</span><span class="p">[</span><span class="s">'default'</span><span class="p">],</span> <span class="s">"mem_rejections"</span><span class="p">,</span> <span class="s">"Rejected writes</span><span class="se">\n</span><span class="s">per sec"</span><span class="p">)</span></code></pre></div>
@@ -205,24 +205,24 @@
<p><img src="/img/YCSB_files/YCSB_7_0.png" alt="png" class="img-responsive" /></p>
-<p>I then re-ran the workload while watching <code class="highlighter-rouge">iostat -dxm 1</code> to see the write rates across all of the disks. I could see that each of the disks was busy in turn, rather than busy in parallel.</p>
+<p>I then re-ran the workload while watching <code>iostat -dxm 1</code> to see the write rates across all of the disks. I could see that each of the disks was busy in turn, rather than busy in parallel.</p>
<p>This reminded me that the default way in which Kudu flushes data is as follows:</p>
-<div class="highlighter-rouge">for each column:
+<pre><code>for each column:
open a new block on disk to write that column, round-robining across disks
iterate over data:
append data to the already-open blocks
for each column:
fsync() the block of data
close the block
-</div>
+</code></pre>
-<p>Because Kudu uses buffered writes, the actual appending of data to the open blocks does not generate immediate IO. Instead, it only dirties pages in the Linux page cache. The actual IO is performed with the <code class="highlighter-rouge">fsync</code> call at the end. Because Kudu defaults to fsyncing each file in turn from a single thread, this was causing the slow performance identified above.</p>
+<p>Because Kudu uses buffered writes, the actual appending of data to the open blocks does not generate immediate IO. Instead, it only dirties pages in the Linux page cache. The actual IO is performed with the <code>fsync</code> call at the end. Because Kudu defaults to fsyncing each file in turn from a single thread, this was causing the slow performance identified above.</p>
-<p>At this point, I consulted with Adar Dembo, who designed much of this code path. He reminded me that we actually have a configuration flag <code class="highlighter-rouge">cfile_do_on_finish=flush</code> which changes the code to something resembling the following:</p>
+<p>At this point, I consulted with Adar Dembo, who designed much of this code path. He reminded me that we actually have a configuration flag <code>cfile_do_on_finish=flush</code> which changes the code to something resembling the following:</p>
-<div class="highlighter-rouge">for each column:
+<pre><code>for each column:
open a new block on disk to write that column, round-robining across disks
iterate over data:
append data to the already-open blocks
@@ -231,16 +231,16 @@ for each column:
for each column:
fsync the block
close the block
-</div>
+</code></pre>
-<p>The <code class="highlighter-rouge">sync_file_range</code> call here asynchronously enqueues the dirty pages to be written back to the disks, and then the following <code class="highlighter-rouge">fsync</code> actually waits for the writeback to be complete. I ran the benchmark for a new configuration with this flag enabled, and plotted the results:</p>
+<p>The <code>sync_file_range</code> call here asynchronously enqueues the dirty pages to be written back to the disks, and then the following <code>fsync</code> actually waits for the writeback to be complete. I ran the benchmark for a new configuration with this flag enabled, and plotted the results:</p>
<div class="highlight"><pre><code class="language-python" data-lang="python"><span class="n">plot_throughput_latency</span><span class="p">(</span><span class="n">data</span><span class="p">[</span><span class="s">'finish=flush'</span><span class="p">])</span></code></pre></div>
<p><img src="/img/YCSB_files/YCSB_9_0.png" alt="png" class="img-responsive" /></p>
-<div class="highlighter-rouge">Average throughput: 52457 ops/sec
-</div>
+<pre><code>Average throughput: 52457 ops/sec
+</code></pre>
<p>This is already a substantial improvement from the default settings. The overall throughput has increased from 31K ops/second to 52K ops/second (<strong>67%</strong>), and we no longer see any dramatic drops in performance or increases in 99th percentile. In fact, the 99th percentile stays comfortably below 1ms for the entire test.</p>
@@ -282,14 +282,14 @@ for each column:
<p>Writing a lot of small flushes compared to a small number of large flushes means that the on-disk data is not as well sorted in the optimized workload. An individual write may need to consult up to 20 bloom filters corresponding to previously flushed pieces of data in order to ensure that it is not an insert with a duplicate primary key.</p>
-<p>So, how can we address this issue? It turns out that the flush threshold is actually configurable with the <code class="highlighter-rouge">flush_threshold_mb</code> flag. I re-ran the workload yet another time with the flush threshold set to 20GB.</p>
+<p>So, how can we address this issue? It turns out that the flush threshold is actually configurable with the <code>flush_threshold_mb</code> flag. I re-ran the workload yet another time with the flush threshold set to 20GB.</p>
<div class="highlight"><pre><code class="language-python" data-lang="python"><span class="n">plot_throughput_latency</span><span class="p">(</span><span class="n">data</span><span class="p">[</span><span class="s">'finish=flush+20GB-threshold'</span><span class="p">])</span></code></pre></div>
<p><img src="/img/YCSB_files/YCSB_17_0.png" alt="png" class="img-responsive" /></p>
-<div class="highlighter-rouge">Average throughput: 67123 ops/sec
-</div>
+<pre><code>Average throughput: 67123 ops/sec
+</code></pre>
<p>This gets us another 28% improvement from 52K ops/second up to 67K ops/second (<strong>+116%</strong> from the default), and we no longer see the troubling downward slope on the throughput graph. Let’s check on the memory and bloom filter metrics again.</p>
@@ -318,13 +318,13 @@ for each column:
<h1 id="tests-with-batched-writes">Tests with Batched Writes</h1>
-<p>The above tests were done with the <code class="highlighter-rouge">sync_ops=true</code> YCSB configuration option. However, we expect that for many heavy write situations, the writers would batch many rows together into larger write operations for better throughput.</p>
+<p>The above tests were done with the <code>sync_ops=true</code> YCSB configuration option. However, we expect that for many heavy write situations, the writers would batch many rows together into larger write operations for better throughput.</p>
-<p>I wanted to ensure that the recommended configuration changes above also improved performance for this workload. So, I re-ran the same experiments, but with YCSB configured to send batches of 100 insert operations to the tablet server using the Kudu client’s <code class="highlighter-rouge">AUTO_FLUSH_BACKGROUND</code> write mode.</p>
+<p>I wanted to ensure that the recommended configuration changes above also improved performance for this workload. So, I re-ran the same experiments, but with YCSB configured to send batches of 100 insert operations to the tablet server using the Kudu client’s <code>AUTO_FLUSH_BACKGROUND</code> write mode.</p>
<p>This time, I compared four configurations:
- the Kudu default settings
-- the defaults, but configured with <code class="highlighter-rouge">cfile_do_on_finish=flush</code> to increase flush IO performance
+- the defaults, but configured with <code>cfile_do_on_finish=flush</code> to increase flush IO performance
- the above, but with the flush thresholds configured to 1G and 10G</p>
<p>For these experiments, we don’t plot latencies, since write latencies are meaningless with batching enabled.</p>
@@ -341,8 +341,8 @@ for each column:
<p><img src="/img/YCSB_files/YCSB_23_1.png" alt="png" class="img-responsive" /></p>
-<div class="highlighter-rouge">Average throughput: 33319 ops/sec
-</div>
+<pre><code>Average throughput: 33319 ops/sec
+</code></pre>
<p><img src="/img/YCSB_files/YCSB_23_3.png" alt="png" class="img-responsive" /></p>
@@ -351,8 +351,8 @@ for each column:
<p><img src="/img/YCSB_files/YCSB_23_5.png" alt="png" class="img-responsive" /></p>
-<div class="highlighter-rouge">Average throughput: 80068 ops/sec
-</div>
+<pre><code>Average throughput: 80068 ops/sec
+</code></pre>
<p><img src="/img/YCSB_files/YCSB_23_7.png" alt="png" class="img-responsive" /></p>
@@ -361,8 +361,8 @@ for each column:
<p><img src="/img/YCSB_files/YCSB_23_9.png" alt="png" class="img-responsive" /></p>
-<div class="highlighter-rouge">Average throughput: 78040 ops/sec
-</div>
+<pre><code>Average throughput: 78040 ops/sec
+</code></pre>
<p><img src="/img/YCSB_files/YCSB_23_11.png" alt="png" class="img-responsive" /></p>
@@ -371,8 +371,8 @@ for each column:
<p><img src="/img/YCSB_files/YCSB_23_13.png" alt="png" class="img-responsive" /></p>
-<div class="highlighter-rouge">Average throughput: 82005 ops/sec
-</div>
+<pre><code>Average throughput: 82005 ops/sec
+</code></pre>
<p><img src="/img/YCSB_files/YCSB_23_15.png" alt="png" class="img-responsive" /></p>
@@ -393,8 +393,8 @@ for each column:
<p>We will likely make these changes in the next Kudu release. In the meantime, users can experiment by adding the following flags to their tablet server configuration:</p>
<ul>
- <li><code class="highlighter-rouge">--cfile_do_on_finish=flush</code></li>
- <li><code class="highlighter-rouge">--flush_threshold_mb=10000</code></li>
+ <li><code>--cfile_do_on_finish=flush</code></li>
+ <li><code>--flush_threshold_mb=10000</code></li>
</ul>
<p>Note that, even if the server hosts many tablets or has less memory than the one used in this test, flushes will still be triggered if the <em>overall</em> memory consumption of the process crosses the configured soft limit. So, configuring a 10GB threshold does not increase the risk of out-of-memory errors.</p>
@@ -418,6 +418,8 @@ for each column:
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2018/10/26/apache-kudu-1-8-0-released.html">Apache Kudu 1.8.0 Released</a> </li>
+
<li> <a href="/2018/09/26/index-skip-scan-optimization-in-kudu.html">Index Skip Scan Optimization in Kudu</a> </li>
<li> <a href="/2018/09/11/simplified-pipelines-with-kudu.html">Simplified Data Pipelines with Kudu</a> </li>
@@ -446,8 +448,6 @@ for each column:
<li> <a href="/2016/11/15/weekly-update.html">Apache Kudu Weekly Update November 15th, 2016</a> </li>
- <li> <a href="/2016/11/01/weekly-update.html">Apache Kudu Weekly Update November 1st, 2016</a> </li>
-
</ul>
</div>
</div>
http://git-wip-us.apache.org/repos/asf/kudu-site/blob/854be1d3/2016/05/03/weekly-update.html
----------------------------------------------------------------------
diff --git a/2016/05/03/weekly-update.html b/2016/05/03/weekly-update.html
index 9f5540a..2cf3c91 100644
--- a/2016/05/03/weekly-update.html
+++ b/2016/05/03/weekly-update.html
@@ -190,6 +190,8 @@ list of conferenace sessions and meetups near you.</li>
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2018/10/26/apache-kudu-1-8-0-released.html">Apache Kudu 1.8.0 Released</a> </li>
+
<li> <a href="/2018/09/26/index-skip-scan-optimization-in-kudu.html">Index Skip Scan Optimization in Kudu</a> </li>
<li> <a href="/2018/09/11/simplified-pipelines-with-kudu.html">Simplified Data Pipelines with Kudu</a> </li>
@@ -218,8 +220,6 @@ list of conferenace sessions and meetups near you.</li>
<li> <a href="/2016/11/15/weekly-update.html">Apache Kudu Weekly Update November 15th, 2016</a> </li>
- <li> <a href="/2016/11/01/weekly-update.html">Apache Kudu Weekly Update November 1st, 2016</a> </li>
-
</ul>
</div>
</div>
http://git-wip-us.apache.org/repos/asf/kudu-site/blob/854be1d3/2016/05/09/weekly-update.html
----------------------------------------------------------------------
diff --git a/2016/05/09/weekly-update.html b/2016/05/09/weekly-update.html
index 77d2fb5..9d77178 100644
--- a/2016/05/09/weekly-update.html
+++ b/2016/05/09/weekly-update.html
@@ -180,6 +180,8 @@ on May 10.</li>
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2018/10/26/apache-kudu-1-8-0-released.html">Apache Kudu 1.8.0 Released</a> </li>
+
<li> <a href="/2018/09/26/index-skip-scan-optimization-in-kudu.html">Index Skip Scan Optimization in Kudu</a> </li>
<li> <a href="/2018/09/11/simplified-pipelines-with-kudu.html">Simplified Data Pipelines with Kudu</a> </li>
@@ -208,8 +210,6 @@ on May 10.</li>
<li> <a href="/2016/11/15/weekly-update.html">Apache Kudu Weekly Update November 15th, 2016</a> </li>
- <li> <a href="/2016/11/01/weekly-update.html">Apache Kudu Weekly Update November 1st, 2016</a> </li>
-
</ul>
</div>
</div>
http://git-wip-us.apache.org/repos/asf/kudu-site/blob/854be1d3/2016/05/16/weekly-update.html
----------------------------------------------------------------------
diff --git a/2016/05/16/weekly-update.html b/2016/05/16/weekly-update.html
index 7fe1a1f..322a6f0 100644
--- a/2016/05/16/weekly-update.html
+++ b/2016/05/16/weekly-update.html
@@ -215,6 +215,8 @@ meetup.</li>
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2018/10/26/apache-kudu-1-8-0-released.html">Apache Kudu 1.8.0 Released</a> </li>
+
<li> <a href="/2018/09/26/index-skip-scan-optimization-in-kudu.html">Index Skip Scan Optimization in Kudu</a> </li>
<li> <a href="/2018/09/11/simplified-pipelines-with-kudu.html">Simplified Data Pipelines with Kudu</a> </li>
@@ -243,8 +245,6 @@ meetup.</li>
<li> <a href="/2016/11/15/weekly-update.html">Apache Kudu Weekly Update November 15th, 2016</a> </li>
- <li> <a href="/2016/11/01/weekly-update.html">Apache Kudu Weekly Update November 1st, 2016</a> </li>
-
</ul>
</div>
</div>
http://git-wip-us.apache.org/repos/asf/kudu-site/blob/854be1d3/2016/05/23/weekly-update.html
----------------------------------------------------------------------
diff --git a/2016/05/23/weekly-update.html b/2016/05/23/weekly-update.html
index d4259e9..fadcfe3 100644
--- a/2016/05/23/weekly-update.html
+++ b/2016/05/23/weekly-update.html
@@ -149,9 +149,9 @@ first release candidate.</p>
</li>
<li>
<p>Since Kudu’s initial release, one of the most commonly requested features
-has been support for the <code class="highlighter-rouge">UPSERT</code> operation. <code class="highlighter-rouge">UPSERT</code>, known in some other
-databases as <code class="highlighter-rouge">INSERT ... ON DUPLICATE KEY UPDATE</code>. This operation has the
-semantics of an <code class="highlighter-rouge">INSERT</code> if no key already exists with the provided primary
+has been support for the <code>UPSERT</code> operation. <code>UPSERT</code>, known in some other
+databases as <code>INSERT ... ON DUPLICATE KEY UPDATE</code>. This operation has the
+semantics of an <code>INSERT</code> if no key already exists with the provided primary
key. Otherwise, it replaces the existing row with the new values.</p>
<p>This week, several developers collaborated to add support for this operation.
@@ -204,6 +204,8 @@ Cloudera User Group</a>.</li>
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2018/10/26/apache-kudu-1-8-0-released.html">Apache Kudu 1.8.0 Released</a> </li>
+
<li> <a href="/2018/09/26/index-skip-scan-optimization-in-kudu.html">Index Skip Scan Optimization in Kudu</a> </li>
<li> <a href="/2018/09/11/simplified-pipelines-with-kudu.html">Simplified Data Pipelines with Kudu</a> </li>
@@ -232,8 +234,6 @@ Cloudera User Group</a>.</li>
<li> <a href="/2016/11/15/weekly-update.html">Apache Kudu Weekly Update November 15th, 2016</a> </li>
- <li> <a href="/2016/11/01/weekly-update.html">Apache Kudu Weekly Update November 1st, 2016</a> </li>
-
</ul>
</div>
</div>
http://git-wip-us.apache.org/repos/asf/kudu-site/blob/854be1d3/2016/06/01/weekly-update.html
----------------------------------------------------------------------
diff --git a/2016/06/01/weekly-update.html b/2016/06/01/weekly-update.html
index 40fbd88..8238755 100644
--- a/2016/06/01/weekly-update.html
+++ b/2016/06/01/weekly-update.html
@@ -170,6 +170,8 @@ hadoop-common test jar. This solved build issues while also removing a nasty dep
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2018/10/26/apache-kudu-1-8-0-released.html">Apache Kudu 1.8.0 Released</a> </li>
+
<li> <a href="/2018/09/26/index-skip-scan-optimization-in-kudu.html">Index Skip Scan Optimization in Kudu</a> </li>
<li> <a href="/2018/09/11/simplified-pipelines-with-kudu.html">Simplified Data Pipelines with Kudu</a> </li>
@@ -198,8 +200,6 @@ hadoop-common test jar. This solved build issues while also removing a nasty dep
<li> <a href="/2016/11/15/weekly-update.html">Apache Kudu Weekly Update November 15th, 2016</a> </li>
- <li> <a href="/2016/11/01/weekly-update.html">Apache Kudu Weekly Update November 1st, 2016</a> </li>
-
</ul>
</div>
</div>
http://git-wip-us.apache.org/repos/asf/kudu-site/blob/854be1d3/2016/06/02/no-default-partitioning.html
----------------------------------------------------------------------
diff --git a/2016/06/02/no-default-partitioning.html b/2016/06/02/no-default-partitioning.html
index 2e3a9c9..fd9790e 100644
--- a/2016/06/02/no-default-partitioning.html
+++ b/2016/06/02/no-default-partitioning.html
@@ -161,7 +161,7 @@ advanced configurations.</p>
<h1 id="c-client">C++ Client</h1>
<p>With the C++ client, creating a new table with hash partitions is as simple as
-calling <code class="highlighter-rouge">KuduTableCreator:add_hash_partitions</code> with the columns to hash and the
+calling <code>KuduTableCreator:add_hash_partitions</code> with the columns to hash and the
number of buckets to use:</p>
<div class="highlight"><pre><code class="language-cpp" data-lang="cpp"><span class="n">unique_ptr</span><span class="o"><</span><span class="n">KuduTableCreator</span><span class="o">></span> <span class="n">table_creator</span><span class="p">(</span><span class="n">my_client</span><span class="o">-></span><span class="n">NewTableCreator</span><span class="p">());</span>
@@ -182,14 +182,14 @@ number of buckets to use:</p>
<span class="n">myClient</span><span class="o">.</span><span class="na">createTable</span><span class="o">(</span><span class="s">"my-table"</span><span class="o">,</span> <span class="n">my_schema</span><span class="o">,</span> <span class="n">options</span><span class="o">);</span></code></pre></div>
<p>In the examples above, if the hash partition configuration is omitted the create
-table operation will fail with the error <code class="highlighter-rouge">Table partitioning must be specified
+table operation will fail with the error <code>Table partitioning must be specified
using setRangePartitionColumns or addHashPartitions</code>. In the Java client this
-manifests as a thrown <code class="highlighter-rouge">IllegalArgumentException</code>, while in the C++ client it is
-returned as a <code class="highlighter-rouge">Status::InvalidArgument</code>.</p>
+manifests as a thrown <code>IllegalArgumentException</code>, while in the C++ client it is
+returned as a <code>Status::InvalidArgument</code>.</p>
<h1 id="impala">Impala</h1>
-<p>When creating Kudu tables with Impala, the formerly optional <code class="highlighter-rouge">DISTRIBUTE BY</code>
+<p>When creating Kudu tables with Impala, the formerly optional <code>DISTRIBUTE BY</code>
clause is now required:</p>
<div class="highlight"><pre><code class="language-sql" data-lang="sql"><span class="k">CREATE</span> <span class="k">TABLE</span> <span class="n">my_table</span> <span class="p">(</span><span class="n">key_column_a</span> <span class="n">STRING</span><span class="p">,</span> <span class="n">key_column_b</span> <span class="n">STRING</span><span class="p">,</span> <span class="n">other_column</span> <span class="n">STRING</span><span class="p">)</span>
@@ -211,6 +211,8 @@ clause is now required:</p>
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2018/10/26/apache-kudu-1-8-0-released.html">Apache Kudu 1.8.0 Released</a> </li>
+
<li> <a href="/2018/09/26/index-skip-scan-optimization-in-kudu.html">Index Skip Scan Optimization in Kudu</a> </li>
<li> <a href="/2018/09/11/simplified-pipelines-with-kudu.html">Simplified Data Pipelines with Kudu</a> </li>
@@ -239,8 +241,6 @@ clause is now required:</p>
<li> <a href="/2016/11/15/weekly-update.html">Apache Kudu Weekly Update November 15th, 2016</a> </li>
- <li> <a href="/2016/11/01/weekly-update.html">Apache Kudu Weekly Update November 1st, 2016</a> </li>
-
</ul>
</div>
</div>
http://git-wip-us.apache.org/repos/asf/kudu-site/blob/854be1d3/2016/06/06/weekly-update.html
----------------------------------------------------------------------
diff --git a/2016/06/06/weekly-update.html b/2016/06/06/weekly-update.html
index 34105e3..0f5cf61 100644
--- a/2016/06/06/weekly-update.html
+++ b/2016/06/06/weekly-update.html
@@ -165,6 +165,8 @@ patches in for the <a href="https://gerrit.cloudera.org/#/c/2642/">Replay Cache<
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2018/10/26/apache-kudu-1-8-0-released.html">Apache Kudu 1.8.0 Released</a> </li>
+
<li> <a href="/2018/09/26/index-skip-scan-optimization-in-kudu.html">Index Skip Scan Optimization in Kudu</a> </li>
<li> <a href="/2018/09/11/simplified-pipelines-with-kudu.html">Simplified Data Pipelines with Kudu</a> </li>
@@ -193,8 +195,6 @@ patches in for the <a href="https://gerrit.cloudera.org/#/c/2642/">Replay Cache<
<li> <a href="/2016/11/15/weekly-update.html">Apache Kudu Weekly Update November 15th, 2016</a> </li>
- <li> <a href="/2016/11/01/weekly-update.html">Apache Kudu Weekly Update November 1st, 2016</a> </li>
-
</ul>
</div>
</div>
http://git-wip-us.apache.org/repos/asf/kudu-site/blob/854be1d3/2016/06/10/apache-kudu-0-9-0-released.html
----------------------------------------------------------------------
diff --git a/2016/06/10/apache-kudu-0-9-0-released.html b/2016/06/10/apache-kudu-0-9-0-released.html
index 00bd3b1..2cda340 100644
--- a/2016/06/10/apache-kudu-0-9-0-released.html
+++ b/2016/06/10/apache-kudu-0-9-0-released.html
@@ -140,6 +140,8 @@ the specification of a partitioning scheme for new tables.</p>
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2018/10/26/apache-kudu-1-8-0-released.html">Apache Kudu 1.8.0 Released</a> </li>
+
<li> <a href="/2018/09/26/index-skip-scan-optimization-in-kudu.html">Index Skip Scan Optimization in Kudu</a> </li>
<li> <a href="/2018/09/11/simplified-pipelines-with-kudu.html">Simplified Data Pipelines with Kudu</a> </li>
@@ -168,8 +170,6 @@ the specification of a partitioning scheme for new tables.</p>
<li> <a href="/2016/11/15/weekly-update.html">Apache Kudu Weekly Update November 15th, 2016</a> </li>
- <li> <a href="/2016/11/01/weekly-update.html">Apache Kudu Weekly Update November 1st, 2016</a> </li>
-
</ul>
</div>
</div>
http://git-wip-us.apache.org/repos/asf/kudu-site/blob/854be1d3/2016/06/13/weekly-update.html
----------------------------------------------------------------------
diff --git a/2016/06/13/weekly-update.html b/2016/06/13/weekly-update.html
index b051ffa..3612f74 100644
--- a/2016/06/13/weekly-update.html
+++ b/2016/06/13/weekly-update.html
@@ -173,6 +173,8 @@ removal happening in this <a href="https://gerrit.cloudera.org/#/c/3350/">patch<
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2018/10/26/apache-kudu-1-8-0-released.html">Apache Kudu 1.8.0 Released</a> </li>
+
<li> <a href="/2018/09/26/index-skip-scan-optimization-in-kudu.html">Index Skip Scan Optimization in Kudu</a> </li>
<li> <a href="/2018/09/11/simplified-pipelines-with-kudu.html">Simplified Data Pipelines with Kudu</a> </li>
@@ -201,8 +203,6 @@ removal happening in this <a href="https://gerrit.cloudera.org/#/c/3350/">patch<
<li> <a href="/2016/11/15/weekly-update.html">Apache Kudu Weekly Update November 15th, 2016</a> </li>
- <li> <a href="/2016/11/01/weekly-update.html">Apache Kudu Weekly Update November 1st, 2016</a> </li>
-
</ul>
</div>
</div>
http://git-wip-us.apache.org/repos/asf/kudu-site/blob/854be1d3/2016/06/17/raft-consensus-single-node.html
----------------------------------------------------------------------
diff --git a/2016/06/17/raft-consensus-single-node.html b/2016/06/17/raft-consensus-single-node.html
index 6c532c7..cc2b83c 100644
--- a/2016/06/17/raft-consensus-single-node.html
+++ b/2016/06/17/raft-consensus-single-node.html
@@ -143,13 +143,13 @@ implementation was complete.</p>
<p>The Consensus API has the following main responsibilities:</p>
<ol>
- <li>Support acting as a Raft <code class="highlighter-rouge">LEADER</code> and replicate writes to a local
+ <li>Support acting as a Raft <code>LEADER</code> and replicate writes to a local
write-ahead log (WAL) as well as followers in the Raft configuration. For
each operation written to the leader, a Raft implementation must keep track
of how many nodes have written a copy of the operation being replicated, and
whether or not that constitutes a majority. Once a majority of the nodes
have written a copy of the data, it is considered committed.</li>
- <li>Support acting as a Raft <code class="highlighter-rouge">FOLLOWER</code> by accepting writes from the leader and
+ <li>Support acting as a Raft <code>FOLLOWER</code> by accepting writes from the leader and
preparing them to be eventually committed.</li>
<li>Support voting in and initiating leader elections.</li>
<li>Support participating in and initiating configuration changes (such as going
@@ -215,6 +215,8 @@ dissertation, which you can find linked from the above web site.</p>
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2018/10/26/apache-kudu-1-8-0-released.html">Apache Kudu 1.8.0 Released</a> </li>
+
<li> <a href="/2018/09/26/index-skip-scan-optimization-in-kudu.html">Index Skip Scan Optimization in Kudu</a> </li>
<li> <a href="/2018/09/11/simplified-pipelines-with-kudu.html">Simplified Data Pipelines with Kudu</a> </li>
@@ -243,8 +245,6 @@ dissertation, which you can find linked from the above web site.</p>
<li> <a href="/2016/11/15/weekly-update.html">Apache Kudu Weekly Update November 15th, 2016</a> </li>
- <li> <a href="/2016/11/01/weekly-update.html">Apache Kudu Weekly Update November 1st, 2016</a> </li>
-
</ul>
</div>
</div>
http://git-wip-us.apache.org/repos/asf/kudu-site/blob/854be1d3/2016/06/21/weekly-update.html
----------------------------------------------------------------------
diff --git a/2016/06/21/weekly-update.html b/2016/06/21/weekly-update.html
index e152b57..0b6b40b 100644
--- a/2016/06/21/weekly-update.html
+++ b/2016/06/21/weekly-update.html
@@ -134,7 +134,7 @@ leveraging the tablets cache.</p>
<li>
<p>In the context of making multi-master reliable in 1.0, Adar Dembo posted a <a href="https://gerrit.cloudera.org/#/c/3393/">design document</a>
on how to handle permanent master failures. Currently the master’s code is missing some features
-like <code class="highlighter-rouge">remote bootstrap</code> which makes it possible for a new replica to download a snapshot of the data
+like <code>remote bootstrap</code> which makes it possible for a new replica to download a snapshot of the data
from the leader replica.</p>
</li>
<li>
@@ -166,6 +166,8 @@ a future post.</p>
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2018/10/26/apache-kudu-1-8-0-released.html">Apache Kudu 1.8.0 Released</a> </li>
+
<li> <a href="/2018/09/26/index-skip-scan-optimization-in-kudu.html">Index Skip Scan Optimization in Kudu</a> </li>
<li> <a href="/2018/09/11/simplified-pipelines-with-kudu.html">Simplified Data Pipelines with Kudu</a> </li>
@@ -194,8 +196,6 @@ a future post.</p>
<li> <a href="/2016/11/15/weekly-update.html">Apache Kudu Weekly Update November 15th, 2016</a> </li>
- <li> <a href="/2016/11/01/weekly-update.html">Apache Kudu Weekly Update November 1st, 2016</a> </li>
-
</ul>
</div>
</div>
http://git-wip-us.apache.org/repos/asf/kudu-site/blob/854be1d3/2016/06/24/multi-master-1-0-0.html
----------------------------------------------------------------------
diff --git a/2016/06/24/multi-master-1-0-0.html b/2016/06/24/multi-master-1-0-0.html
index 3891256..7286d33 100644
--- a/2016/06/24/multi-master-1-0-0.html
+++ b/2016/06/24/multi-master-1-0-0.html
@@ -144,9 +144,9 @@ can be safely enabled in production clusters.</p>
<p>To use replicated masters, a Kudu operator must deploy some number of Kudu
masters, providing the hostname and port number of each master in the group via
-the <code class="highlighter-rouge">--master_address</code> command line option. For example, each master in a
+the <code>--master_address</code> command line option. For example, each master in a
three-node deployment should be started with
-<code class="highlighter-rouge">--master_address=<host1:port1>,<host2:port2><host3:port3></code>. In Raft parlance,
+<code>--master_address=<host1:port1>,<host2:port2><host3:port3></code>. In Raft parlance,
this group of masters is known as a <em>Raft configuration</em>.</p>
<p>At startup, a Raft configuration of masters will hold a leader election and
@@ -191,7 +191,7 @@ clients are also configured with the locations of all masters. Unlike tablet
servers, they always communicate with the leader master as follower masters will
reject client requests. To do this, clients must determine which master is the
leader before sending the first request as well as whenever any request fails
-with a <code class="highlighter-rouge">NOT_THE_LEADER</code> error.</p>
+with a <code>NOT_THE_LEADER</code> error.</p>
<h1 id="remaining-work-for-kudu-10">Remaining work for Kudu 1.0</h1>
@@ -228,6 +228,8 @@ nothing has been implemented yet. Stay tuned!</p>
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2018/10/26/apache-kudu-1-8-0-released.html">Apache Kudu 1.8.0 Released</a> </li>
+
<li> <a href="/2018/09/26/index-skip-scan-optimization-in-kudu.html">Index Skip Scan Optimization in Kudu</a> </li>
<li> <a href="/2018/09/11/simplified-pipelines-with-kudu.html">Simplified Data Pipelines with Kudu</a> </li>
@@ -256,8 +258,6 @@ nothing has been implemented yet. Stay tuned!</p>
<li> <a href="/2016/11/15/weekly-update.html">Apache Kudu Weekly Update November 15th, 2016</a> </li>
- <li> <a href="/2016/11/01/weekly-update.html">Apache Kudu Weekly Update November 1st, 2016</a> </li>
-
</ul>
</div>
</div>
http://git-wip-us.apache.org/repos/asf/kudu-site/blob/854be1d3/2016/06/27/weekly-update.html
----------------------------------------------------------------------
diff --git a/2016/06/27/weekly-update.html b/2016/06/27/weekly-update.html
index 6f7b17c..4cbc866 100644
--- a/2016/06/27/weekly-update.html
+++ b/2016/06/27/weekly-update.html
@@ -230,6 +230,8 @@ a future post.</p>
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2018/10/26/apache-kudu-1-8-0-released.html">Apache Kudu 1.8.0 Released</a> </li>
+
<li> <a href="/2018/09/26/index-skip-scan-optimization-in-kudu.html">Index Skip Scan Optimization in Kudu</a> </li>
<li> <a href="/2018/09/11/simplified-pipelines-with-kudu.html">Simplified Data Pipelines with Kudu</a> </li>
@@ -258,8 +260,6 @@ a future post.</p>
<li> <a href="/2016/11/15/weekly-update.html">Apache Kudu Weekly Update November 15th, 2016</a> </li>
- <li> <a href="/2016/11/01/weekly-update.html">Apache Kudu Weekly Update November 1st, 2016</a> </li>
-
</ul>
</div>
</div>
http://git-wip-us.apache.org/repos/asf/kudu-site/blob/854be1d3/2016/07/01/apache-kudu-0-9-1-released.html
----------------------------------------------------------------------
diff --git a/2016/07/01/apache-kudu-0-9-1-released.html b/2016/07/01/apache-kudu-0-9-1-released.html
index 101148a..f468513 100644
--- a/2016/07/01/apache-kudu-0-9-1-released.html
+++ b/2016/07/01/apache-kudu-0-9-1-released.html
@@ -138,6 +138,8 @@ of 0.9.0 are encouraged to update to the new version at their earliest convenien
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2018/10/26/apache-kudu-1-8-0-released.html">Apache Kudu 1.8.0 Released</a> </li>
+
<li> <a href="/2018/09/26/index-skip-scan-optimization-in-kudu.html">Index Skip Scan Optimization in Kudu</a> </li>
<li> <a href="/2018/09/11/simplified-pipelines-with-kudu.html">Simplified Data Pipelines with Kudu</a> </li>
@@ -166,8 +168,6 @@ of 0.9.0 are encouraged to update to the new version at their earliest convenien
<li> <a href="/2016/11/15/weekly-update.html">Apache Kudu Weekly Update November 15th, 2016</a> </li>
- <li> <a href="/2016/11/01/weekly-update.html">Apache Kudu Weekly Update November 1st, 2016</a> </li>
-
</ul>
</div>
</div>
http://git-wip-us.apache.org/repos/asf/kudu-site/blob/854be1d3/2016/07/11/weekly-update.html
----------------------------------------------------------------------
diff --git a/2016/07/11/weekly-update.html b/2016/07/11/weekly-update.html
index 09e991e..a377087 100644
--- a/2016/07/11/weekly-update.html
+++ b/2016/07/11/weekly-update.html
@@ -196,6 +196,8 @@ a future post.</p>
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2018/10/26/apache-kudu-1-8-0-released.html">Apache Kudu 1.8.0 Released</a> </li>
+
<li> <a href="/2018/09/26/index-skip-scan-optimization-in-kudu.html">Index Skip Scan Optimization in Kudu</a> </li>
<li> <a href="/2018/09/11/simplified-pipelines-with-kudu.html">Simplified Data Pipelines with Kudu</a> </li>
@@ -224,8 +226,6 @@ a future post.</p>
<li> <a href="/2016/11/15/weekly-update.html">Apache Kudu Weekly Update November 15th, 2016</a> </li>
- <li> <a href="/2016/11/01/weekly-update.html">Apache Kudu Weekly Update November 1st, 2016</a> </li>
-
</ul>
</div>
</div>
http://git-wip-us.apache.org/repos/asf/kudu-site/blob/854be1d3/2016/07/18/weekly-update.html
----------------------------------------------------------------------
diff --git a/2016/07/18/weekly-update.html b/2016/07/18/weekly-update.html
index 217519b..8fdfe22 100644
--- a/2016/07/18/weekly-update.html
+++ b/2016/07/18/weekly-update.html
@@ -188,6 +188,8 @@ a future post.</p>
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2018/10/26/apache-kudu-1-8-0-released.html">Apache Kudu 1.8.0 Released</a> </li>
+
<li> <a href="/2018/09/26/index-skip-scan-optimization-in-kudu.html">Index Skip Scan Optimization in Kudu</a> </li>
<li> <a href="/2018/09/11/simplified-pipelines-with-kudu.html">Simplified Data Pipelines with Kudu</a> </li>
@@ -216,8 +218,6 @@ a future post.</p>
<li> <a href="/2016/11/15/weekly-update.html">Apache Kudu Weekly Update November 15th, 2016</a> </li>
- <li> <a href="/2016/11/01/weekly-update.html">Apache Kudu Weekly Update November 1st, 2016</a> </li>
-
</ul>
</div>
</div>
http://git-wip-us.apache.org/repos/asf/kudu-site/blob/854be1d3/2016/07/25/asf-graduation.html
----------------------------------------------------------------------
diff --git a/2016/07/25/asf-graduation.html b/2016/07/25/asf-graduation.html
index e73d7c3..038d214 100644
--- a/2016/07/25/asf-graduation.html
+++ b/2016/07/25/asf-graduation.html
@@ -164,6 +164,8 @@ Established in 1999, the all-volunteer Foundation oversees more than 350 leading
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2018/10/26/apache-kudu-1-8-0-released.html">Apache Kudu 1.8.0 Released</a> </li>
+
<li> <a href="/2018/09/26/index-skip-scan-optimization-in-kudu.html">Index Skip Scan Optimization in Kudu</a> </li>
<li> <a href="/2018/09/11/simplified-pipelines-with-kudu.html">Simplified Data Pipelines with Kudu</a> </li>
@@ -192,8 +194,6 @@ Established in 1999, the all-volunteer Foundation oversees more than 350 leading
<li> <a href="/2016/11/15/weekly-update.html">Apache Kudu Weekly Update November 15th, 2016</a> </li>
- <li> <a href="/2016/11/01/weekly-update.html">Apache Kudu Weekly Update November 1st, 2016</a> </li>
-
</ul>
</div>
</div>
http://git-wip-us.apache.org/repos/asf/kudu-site/blob/854be1d3/2016/07/26/weekly-update.html
----------------------------------------------------------------------
diff --git a/2016/07/26/weekly-update.html b/2016/07/26/weekly-update.html
index 2194cb1..c9b34e0 100644
--- a/2016/07/26/weekly-update.html
+++ b/2016/07/26/weekly-update.html
@@ -136,8 +136,8 @@ new name and status.</li>
<ul>
<li>
- <p>Dan Burkert contributed a few patches that repackage the Java client under <code class="highlighter-rouge">org.apache.kudu</code>
-in place of <code class="highlighter-rouge">org.kududb</code>. This was done in a <strong>backward-incompatible</strong> way, meaning that import
+ <p>Dan Burkert contributed a few patches that repackage the Java client under <code>org.apache.kudu</code>
+in place of <code>org.kududb</code>. This was done in a <strong>backward-incompatible</strong> way, meaning that import
statements will have to be modified in existing Java code to compile against a newer Kudu JAR
version (from 0.10.0 onward). This stems from <a href="http://mail-archives.apache.org/mod_mbox/kudu-dev/201605.mbox/%3CCAGpTDNcJohQBgjzXafXJQdqmBB4sL495p5V_BJRXk_nAGWbzhA@mail.gmail.com%3E">a discussion</a>
initiated in May. It won’t have an impact on C++ or Python users, and it isn’t affecting wire
@@ -147,12 +147,12 @@ compatibility.</p>
<p>Still on the Java-side, J-D Cryans pushed <a href="https://gerrit.cloudera.org/#/c/3055/">a patch</a>
that completely changes how Exceptions are managed. Before this change, users had to introspect
generic Exception objects, making it a guessing game and discouraging good error handling.
-Now, the synchronous client’s methods throw <code class="highlighter-rouge">KuduException</code> which packages a <code class="highlighter-rouge">Status</code> object
+Now, the synchronous client’s methods throw <code>KuduException</code> which packages a <code>Status</code> object
that can be interrogated. This is very similar to how the C++ API works.</p>
<p>Existing code that uses the new Kudu JAR should still compile since this change replaces generic
-<code class="highlighter-rouge">Exception</code> with a more specific <code class="highlighter-rouge">KuduException</code>. Error handling done by string-matching the
-exception messages should now use the provided <code class="highlighter-rouge">Status</code> object.</p>
+<code>Exception</code> with a more specific <code>KuduException</code>. Error handling done by string-matching the
+exception messages should now use the provided <code>Status</code> object.</p>
</li>
<li>
<p>Alexey Serbin’s <a href="https://gerrit.cloudera.org/#/c/3619/">patch</a> that adds Doxygen-based
@@ -160,11 +160,11 @@ documentation was pushed and the new API documentation for C++ developers will b
with the next release.</p>
</li>
<li>
- <p>Todd has made many improvements to the <code class="highlighter-rouge">ksck</code> tool over the last week. Building upon Will
-Berkeley’s <a href="https://gerrit.cloudera.org/#/c/3632/">WIP patch for KUDU-1516</a>, <code class="highlighter-rouge">ksck</code> can
+ <p>Todd has made many improvements to the <code>ksck</code> tool over the last week. Building upon Will
+Berkeley’s <a href="https://gerrit.cloudera.org/#/c/3632/">WIP patch for KUDU-1516</a>, <code>ksck</code> can
now detect more problematic situations like if a tablet doesn’t have a majority of replicas on
live tablet servers, or if those replicas aren’t in a good state.
-<code class="highlighter-rouge">ksck</code> is also <a href="https://gerrit.cloudera.org/#/c/3705/">now faster</a> when run against a large
+<code>ksck</code> is also <a href="https://gerrit.cloudera.org/#/c/3705/">now faster</a> when run against a large
cluster with a lot of tablets, among other improvements.</p>
</li>
<li>
@@ -196,6 +196,8 @@ a future post.</p>
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2018/10/26/apache-kudu-1-8-0-released.html">Apache Kudu 1.8.0 Released</a> </li>
+
<li> <a href="/2018/09/26/index-skip-scan-optimization-in-kudu.html">Index Skip Scan Optimization in Kudu</a> </li>
<li> <a href="/2018/09/11/simplified-pipelines-with-kudu.html">Simplified Data Pipelines with Kudu</a> </li>
@@ -224,8 +226,6 @@ a future post.</p>
<li> <a href="/2016/11/15/weekly-update.html">Apache Kudu Weekly Update November 15th, 2016</a> </li>
- <li> <a href="/2016/11/01/weekly-update.html">Apache Kudu Weekly Update November 1st, 2016</a> </li>
-
</ul>
</div>
</div>
http://git-wip-us.apache.org/repos/asf/kudu-site/blob/854be1d3/2016/08/08/weekly-update.html
----------------------------------------------------------------------
diff --git a/2016/08/08/weekly-update.html b/2016/08/08/weekly-update.html
index 807d0c2..90091cc 100644
--- a/2016/08/08/weekly-update.html
+++ b/2016/08/08/weekly-update.html
@@ -195,6 +195,8 @@ a future post.</p>
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2018/10/26/apache-kudu-1-8-0-released.html">Apache Kudu 1.8.0 Released</a> </li>
+
<li> <a href="/2018/09/26/index-skip-scan-optimization-in-kudu.html">Index Skip Scan Optimization in Kudu</a> </li>
<li> <a href="/2018/09/11/simplified-pipelines-with-kudu.html">Simplified Data Pipelines with Kudu</a> </li>
@@ -223,8 +225,6 @@ a future post.</p>
<li> <a href="/2016/11/15/weekly-update.html">Apache Kudu Weekly Update November 15th, 2016</a> </li>
- <li> <a href="/2016/11/01/weekly-update.html">Apache Kudu Weekly Update November 1st, 2016</a> </li>
-
</ul>
</div>
</div>
http://git-wip-us.apache.org/repos/asf/kudu-site/blob/854be1d3/2016/08/16/weekly-update.html
----------------------------------------------------------------------
diff --git a/2016/08/16/weekly-update.html b/2016/08/16/weekly-update.html
index f3a0523..69ba766 100644
--- a/2016/08/16/weekly-update.html
+++ b/2016/08/16/weekly-update.html
@@ -168,20 +168,20 @@ from a single-master configuration to a multi-master one. Along the way, he
started building some common infrastructure for command-line tooling.</p>
<p>Since Kudu’s initial release, it has included separate binaries for different
-administrative or operational tools (e.g. <code class="highlighter-rouge">kudu-ts-cli</code>, <code class="highlighter-rouge">kudu-ksck</code>, <code class="highlighter-rouge">kudu-fs_dump</code>,
-<code class="highlighter-rouge">log-dump</code>, etc). Despite having similar usage, these tools don’t share much code,
+administrative or operational tools (e.g. <code>kudu-ts-cli</code>, <code>kudu-ksck</code>, <code>kudu-fs_dump</code>,
+<code>log-dump</code>, etc). Despite having similar usage, these tools don’t share much code,
and the separate statically linked binaries make the Kudu packages take more disk
space than strictly necessary.</p>
- <p>Adar’s work has introduced a new top-level <code class="highlighter-rouge">kudu</code> binary which exposes a set of subcommands,
-much like the <code class="highlighter-rouge">git</code> and <code class="highlighter-rouge">docker</code> binaries with which readers may be familiar.
+ <p>Adar’s work has introduced a new top-level <code>kudu</code> binary which exposes a set of subcommands,
+much like the <code>git</code> and <code>docker</code> binaries with which readers may be familiar.
For example, a new tool he has built for dumping peer identifiers from a tablet’s
-consensus metadata is triggered using <code class="highlighter-rouge">kudu tablet cmeta print_replica_uuids</code>.</p>
+consensus metadata is triggered using <code>kudu tablet cmeta print_replica_uuids</code>.</p>
<p>This new tool will be available in the upcoming 0.10.0 release; however, migration
of the existing tools to the new infrastructure has not yet been completed. We
expect that by Kudu 1.0, the old tools will be removed in favor of more subcommands
-of the <code class="highlighter-rouge">kudu</code> tool.</p>
+of the <code>kudu</code> tool.</p>
</li>
<li>
<p>Todd Lipcon picked up the work started by David Alves in July to provide
@@ -205,7 +205,7 @@ was <a href="https://gerrit.cloudera.org/#/c/3840/">committed</a> this week. The
docs will be published as part of the 0.10.0 release.</p>
</li>
<li>
- <p>Alexey also continued work on implementing the <code class="highlighter-rouge">AUTO_FLUSH_BACKGROUND</code> write
+ <p>Alexey also continued work on implementing the <code>AUTO_FLUSH_BACKGROUND</code> write
mode for the C++ client. This feature makes it easier to implement high-throughput
ingest using the C++ API by automatically handling the batching and flushing of writes
based on a configurable buffer size.</p>
@@ -245,6 +245,8 @@ a future post.</p>
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2018/10/26/apache-kudu-1-8-0-released.html">Apache Kudu 1.8.0 Released</a> </li>
+
<li> <a href="/2018/09/26/index-skip-scan-optimization-in-kudu.html">Index Skip Scan Optimization in Kudu</a> </li>
<li> <a href="/2018/09/11/simplified-pipelines-with-kudu.html">Simplified Data Pipelines with Kudu</a> </li>
@@ -273,8 +275,6 @@ a future post.</p>
<li> <a href="/2016/11/15/weekly-update.html">Apache Kudu Weekly Update November 15th, 2016</a> </li>
- <li> <a href="/2016/11/01/weekly-update.html">Apache Kudu Weekly Update November 1st, 2016</a> </li>
-
</ul>
</div>
</div>
http://git-wip-us.apache.org/repos/asf/kudu-site/blob/854be1d3/2016/08/23/apache-kudu-0-10-0-released.html
----------------------------------------------------------------------
diff --git a/2016/08/23/apache-kudu-0-10-0-released.html b/2016/08/23/apache-kudu-0-10-0-released.html
index 7cfa6be..6e1e3d6 100644
--- a/2016/08/23/apache-kudu-0-10-0-released.html
+++ b/2016/08/23/apache-kudu-0-10-0-released.html
@@ -161,6 +161,8 @@ the release notes below.</p>
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2018/10/26/apache-kudu-1-8-0-released.html">Apache Kudu 1.8.0 Released</a> </li>
+
<li> <a href="/2018/09/26/index-skip-scan-optimization-in-kudu.html">Index Skip Scan Optimization in Kudu</a> </li>
<li> <a href="/2018/09/11/simplified-pipelines-with-kudu.html">Simplified Data Pipelines with Kudu</a> </li>
@@ -189,8 +191,6 @@ the release notes below.</p>
<li> <a href="/2016/11/15/weekly-update.html">Apache Kudu Weekly Update November 15th, 2016</a> </li>
- <li> <a href="/2016/11/01/weekly-update.html">Apache Kudu Weekly Update November 1st, 2016</a> </li>
-
</ul>
</div>
</div>
http://git-wip-us.apache.org/repos/asf/kudu-site/blob/854be1d3/2016/08/23/new-range-partitioning-features.html
----------------------------------------------------------------------
diff --git a/2016/08/23/new-range-partitioning-features.html b/2016/08/23/new-range-partitioning-features.html
index cb0065a..c8353a4 100644
--- a/2016/08/23/new-range-partitioning-features.html
+++ b/2016/08/23/new-range-partitioning-features.html
@@ -221,6 +221,8 @@ be updated to 0.10.</p>
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2018/10/26/apache-kudu-1-8-0-released.html">Apache Kudu 1.8.0 Released</a> </li>
+
<li> <a href="/2018/09/26/index-skip-scan-optimization-in-kudu.html">Index Skip Scan Optimization in Kudu</a> </li>
<li> <a href="/2018/09/11/simplified-pipelines-with-kudu.html">Simplified Data Pipelines with Kudu</a> </li>
@@ -249,8 +251,6 @@ be updated to 0.10.</p>
<li> <a href="/2016/11/15/weekly-update.html">Apache Kudu Weekly Update November 15th, 2016</a> </li>
- <li> <a href="/2016/11/01/weekly-update.html">Apache Kudu Weekly Update November 1st, 2016</a> </li>
-
</ul>
</div>
</div>
http://git-wip-us.apache.org/repos/asf/kudu-site/blob/854be1d3/2016/08/31/intro-flume-kudu-sink.html
----------------------------------------------------------------------
diff --git a/2016/08/31/intro-flume-kudu-sink.html b/2016/08/31/intro-flume-kudu-sink.html
index 7f94bc9..d9b0491 100644
--- a/2016/08/31/intro-flume-kudu-sink.html
+++ b/2016/08/31/intro-flume-kudu-sink.html
@@ -225,7 +225,7 @@ release and the source code can be found <a href="https://github.com/apache/kudu
<p>Here is a sample flume configuration file:</p>
-<div class="highlighter-rouge">agent1.sources = source1
+<pre><code>agent1.sources = source1
agent1.channels = channel1
agent1.sinks = sink1
@@ -243,25 +243,25 @@ agent1.sinks.sink1.tableName = stats
agent1.sinks.sink1.channel = channel1
agent1.sinks.sink1.batchSize = 50
agent1.sinks.sink1.producer = org.apache.kudu.flume.sink.SimpleKuduEventProducer
-</div>
+</code></pre>
-<p>We define a source called <code class="highlighter-rouge">source1</code> which simply executes a <code class="highlighter-rouge">vmstat</code> command to continuously generate
-virtual memory statistics for the machine and queue events into an in-memory <code class="highlighter-rouge">channel1</code> channel,
-which in turn is used for writing these events to a Kudu table called <code class="highlighter-rouge">stats</code>. We are using
-<code class="highlighter-rouge">org.apache.kudu.flume.sink.SimpleKuduEventProducer</code> as the producer. <code class="highlighter-rouge">SimpleKuduEventProducer</code> is
+<p>We define a source called <code>source1</code> which simply executes a <code>vmstat</code> command to continuously generate
+virtual memory statistics for the machine and queue events into an in-memory <code>channel1</code> channel,
+which in turn is used for writing these events to a Kudu table called <code>stats</code>. We are using
+<code>org.apache.kudu.flume.sink.SimpleKuduEventProducer</code> as the producer. <code>SimpleKuduEventProducer</code> is
the built-in and default producer, but it’s implemented as a showcase for how to write Flume
events into Kudu tables. For any serious functionality we’d have to write a custom producer. We
-need to make this producer and the <code class="highlighter-rouge">KuduSink</code> class available to Flume. We can do that by simply
-copying the <code class="highlighter-rouge">kudu-flume-sink-<VERSION>.jar</code> jar file from the Kudu distribution to the
-<code class="highlighter-rouge">$FLUME_HOME/plugins.d/kudu-sink/lib</code> directory in the Flume installation. The jar file contains
-<code class="highlighter-rouge">KuduSink</code> and all of its dependencies (including Kudu java client classes).</p>
+need to make this producer and the <code>KuduSink</code> class available to Flume. We can do that by simply
+copying the <code>kudu-flume-sink-<VERSION>.jar</code> jar file from the Kudu distribution to the
+<code>$FLUME_HOME/plugins.d/kudu-sink/lib</code> directory in the Flume installation. The jar file contains
+<code>KuduSink</code> and all of its dependencies (including Kudu java client classes).</p>
<p>At a minimum, the Kudu Flume Sink needs to know where the Kudu masters are
-(<code class="highlighter-rouge">agent1.sinks.sink1.masterAddresses = localhost</code>) and which Kudu table should be used for writing
-Flume events to (<code class="highlighter-rouge">agent1.sinks.sink1.tableName = stats</code>). The Kudu Flume Sink doesn’t create this
+(<code>agent1.sinks.sink1.masterAddresses = localhost</code>) and which Kudu table should be used for writing
+Flume events to (<code>agent1.sinks.sink1.tableName = stats</code>). The Kudu Flume Sink doesn’t create this
table, it has to be created before the Kudu Flume Sink is started.</p>
-<p>You may also notice the <code class="highlighter-rouge">batchSize</code> parameter. Batch size is used for batching up to that many
+<p>You may also notice the <code>batchSize</code> parameter. Batch size is used for batching up to that many
Flume events and flushing the entire batch in one shot. Tuning batchSize properly can have a huge
impact on ingest performance of the Kudu cluster.</p>
@@ -311,89 +311,89 @@ impact on ingest performance of the Kudu cluster.</p>
<p>Let’s take a look at the source code for the built-in producer class:</p>
-<div class="highlighter-rouge"><span class="kd">public</span> <span class="kd">class</span> <span class="nc">SimpleKuduEventProducer</span> <span class="kd">implements</span> <span class="n">KuduEventProducer</span> <span class="o">{</span>
- <span class="kd">private</span> <span class="kt">byte</span><span class="o">[]</span> <span class="n">payload</span><span class="o">;</span>
- <span class="kd">private</span> <span class="n">KuduTable</span> <span class="n">table</span><span class="o">;</span>
- <span class="kd">private</span> <span class="n">String</span> <span class="n">payloadColumn</span><span class="o">;</span>
-
- <span class="kd">public</span> <span class="nf">SimpleKuduEventProducer</span><span class="o">(){</span>
- <span class="o">}</span>
-
- <span class="nd">@Override</span>
- <span class="kd">public</span> <span class="kt">void</span> <span class="nf">configure</span><span class="o">(</span><span class="n">Context</span> <span class="n">context</span><span class="o">)</span> <span class="o">{</span>
- <span class="n">payloadColumn</span> <span class="o">=</span> <span class="n">context</span><span class="o">.</span><span class="na">getString</span><span class="o">(</span><span class="s">"payloadColumn"</span><span class="o">,</span><span class="s">"payload"</span><span class="o">);</span>
- <span class="o">}</span>
-
- <span class="nd">@Override</span>
- <span class="kd">public</span> <span class="kt">void</span> <span class="nf">configure</span><span class="o">(</span><span class="n">ComponentConfiguration</span> <span class="n">conf</span><span class="o">)</span> <span class="o">{</span>
- <span class="o">}</span>
-
- <span class="nd">@Override</span>
- <span class="kd">public</span> <span class="kt">void</span> <span class="nf">initialize</span><span class="o">(</span><span class="n">Event</span> <span class="n">event</span><span class="o">,</span> <span class="n">KuduTable</span> <span class="n">table</span><span class="o">)</span> <span class="o">{</span>
- <span class="k">this</span><span class="o">.</span><span class="na">payload</span> <span class="o">=</span> <span class="n">event</span><span class="o">.</span><span class="na">getBody</span><span class="o">();</span>
- <span class="k">this</span><span class="o">.</span><span class="na">table</span> <span class="o">=</span> <span class="n">table</span><span class="o">;</span>
- <span class="o">}</span>
-
- <span class="nd">@Override</span>
- <span class="kd">public</span> <span class="n">List</span><span class="o"><</span><span class="n">Operation</span><span class="o">></span> <span class="nf">getOperations</span><span class="o">()</span> <span class="kd">throws</span> <span class="n">FlumeException</span> <span class="o">{</span>
- <span class="k">try</span> <span class="o">{</span>
- <span class="n">Insert</span> <span class="n">insert</span> <span class="o">=</span> <span class="n">table</span><span class="o">.</span><span class="na">newInsert</span><span class="o">();</span>
- <span class="n">PartialRow</span> <span class="n">row</span> <span class="o">=</span> <span class="n">insert</span><span class="o">.</span><span class="na">getRow</span><span class="o">();</span>
- <span class="n">row</span><span class="o">.</span><span class="na">addBinary</span><span class="o">(</span><span class="n">payloadColumn</span><span class="o">,</span> <span class="n">payload</span><span class="o">);</span>
-
- <span class="k">return</span> <span class="n">Collections</span><span class="o">.</span><span class="na">singletonList</span><span class="o">((</span><span class="n">Operation</span><span class="o">)</span> <span class="n">insert</span><span class="o">);</span>
- <span class="o">}</span> <span class="k">catch</span> <span class="o">(</span><span class="n">Exception</span> <span class="n">e</span><span class="o">){</span>
- <span class="k">throw</span> <span class="k">new</span> <span class="nf">FlumeException</span><span class="o">(</span><span class="s">"Failed to create Kudu Insert object!"</span><span class="o">,</span> <span class="n">e</span><span class="o">);</span>
- <span class="o">}</span>
- <span class="o">}</span>
-
- <span class="nd">@Override</span>
- <span class="kd">public</span> <span class="kt">void</span> <span class="nf">close</span><span class="o">()</span> <span class="o">{</span>
- <span class="o">}</span>
-<span class="o">}</span>
-</div>
-
-<p><code class="highlighter-rouge">SimpleKuduEventProducer</code> implements the <code class="highlighter-rouge">org.apache.kudu.flume.sink.KuduEventProducer</code> interface,
+<pre><code class="language-java">public class SimpleKuduEventProducer implements KuduEventProducer {
+ private byte[] payload;
+ private KuduTable table;
+ private String payloadColumn;
+
+ public SimpleKuduEventProducer(){
+ }
+
+ @Override
+ public void configure(Context context) {
+ payloadColumn = context.getString("payloadColumn","payload");
+ }
+
+ @Override
+ public void configure(ComponentConfiguration conf) {
+ }
+
+ @Override
+ public void initialize(Event event, KuduTable table) {
+ this.payload = event.getBody();
+ this.table = table;
+ }
+
+ @Override
+ public List<Operation> getOperations() throws FlumeException {
+ try {
+ Insert insert = table.newInsert();
+ PartialRow row = insert.getRow();
+ row.addBinary(payloadColumn, payload);
+
+ return Collections.singletonList((Operation) insert);
+ } catch (Exception e){
+ throw new FlumeException("Failed to create Kudu Insert object!", e);
+ }
+ }
+
+ @Override
+ public void close() {
+ }
+}
+</code></pre>
+
+<p><code>SimpleKuduEventProducer</code> implements the <code>org.apache.kudu.flume.sink.KuduEventProducer</code> interface,
which itself looks like this:</p>
-<div class="highlighter-rouge"><span class="kd">public</span> <span class="kd">interface</span> <span class="nc">KuduEventProducer</span> <span class="kd">extends</span> <span class="n">Configurable</span><span class="o">,</span> <span class="n">ConfigurableComponent</span> <span class="o">{</span>
- <span class="cm">/**
+<pre><code class="language-java">public interface KuduEventProducer extends Configurable, ConfigurableComponent {
+ /**
* Initialize the event producer.
* @param event to be written to Kudu
* @param table the KuduTable object used for creating Kudu Operation objects
- */</span>
- <span class="kt">void</span> <span class="nf">initialize</span><span class="o">(</span><span class="n">Event</span> <span class="n">event</span><span class="o">,</span> <span class="n">KuduTable</span> <span class="n">table</span><span class="o">);</span>
+ */
+ void initialize(Event event, KuduTable table);
- <span class="cm">/**
+ /**
* Get the operations that should be written out to Kudu as a result of this
* event. This list is written to Kudu using the Kudu client API.
* @return List of {@link org.kududb.client.Operation} which
* are written as such to Kudu
- */</span>
- <span class="n">List</span><span class="o"><</span><span class="n">Operation</span><span class="o">></span> <span class="nf">getOperations</span><span class="o">();</span>
+ */
+ List<Operation> getOperations();
- <span class="cm">/*
+ /*
* Clean up any state. This will be called when the sink is being stopped.
- */</span>
- <span class="kt">void</span> <span class="nf">close</span><span class="o">();</span>
-<span class="o">}</span>
-</div>
+ */
+ void close();
+}
+</code></pre>
-<p><code class="highlighter-rouge">public void configure(Context context)</code> is called when an instance of our producer is instantiated
+<p><code>public void configure(Context context)</code> is called when an instance of our producer is instantiated
by the KuduSink. SimpleKuduEventProducer’s implementation looks for a producer parameter named
-<code class="highlighter-rouge">payloadColumn</code> and uses its value (“payload” if not overridden in Flume configuration file) as the
+<code>payloadColumn</code> and uses its value (“payload” if not overridden in Flume configuration file) as the
column which will hold the value of the Flume event payload. If you recall from above, we had
-configured the KuduSink to listen for events generated from the <code class="highlighter-rouge">vmstat</code> command. Each output row
-from that command will be stored as a new row containing a <code class="highlighter-rouge">payload</code> column in the <code class="highlighter-rouge">stats</code> table.
-<code class="highlighter-rouge">SimpleKuduEventProducer</code> does not have any configuration parameters, but if it had any we would
-define them by prefixing it with <code class="highlighter-rouge">producer.</code> (<code class="highlighter-rouge">agent1.sinks.sink1.producer.parameter1</code> for
+configured the KuduSink to listen for events generated from the <code>vmstat</code> command. Each output row
+from that command will be stored as a new row containing a <code>payload</code> column in the <code>stats</code> table.
+<code>SimpleKuduEventProducer</code> does not have any configuration parameters, but if it had any we would
+define them by prefixing it with <code>producer.</code> (<code>agent1.sinks.sink1.producer.parameter1</code> for
example).</p>
-<p>The main producer logic resides in the <code class="highlighter-rouge">public List<Operation> getOperations()</code> method. In
+<p>The main producer logic resides in the <code>public List<Operation> getOperations()</code> method. In
SimpleKuduEventProducer’s implementation we simply insert the binary body of the Flume event into
-the Kudu table. Here we call Kudu’s <code class="highlighter-rouge">newInsert()</code> to initiate an insert, but could have used
-<code class="highlighter-rouge">Upsert</code> if updating an existing row was also an option, in fact there’s another producer
-implementation available for doing just that: <code class="highlighter-rouge">SimpleKeyedKuduEventProducer</code>. Most probably you
+the Kudu table. Here we call Kudu’s <code>newInsert()</code> to initiate an insert, but could have used
+<code>Upsert</code> if updating an existing row was also an option, in fact there’s another producer
+implementation available for doing just that: <code>SimpleKeyedKuduEventProducer</code>. Most probably you
will need to write your own custom producer in the real world, but you can base your implementation
on the built-in ones.</p>
@@ -423,6 +423,8 @@ is included in the Kudu distribution. You can follow him on Twitter at
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2018/10/26/apache-kudu-1-8-0-released.html">Apache Kudu 1.8.0 Released</a> </li>
+
<li> <a href="/2018/09/26/index-skip-scan-optimization-in-kudu.html">Index Skip Scan Optimization in Kudu</a> </li>
<li> <a href="/2018/09/11/simplified-pipelines-with-kudu.html">Simplified Data Pipelines with Kudu</a> </li>
@@ -451,8 +453,6 @@ is included in the Kudu distribution. You can follow him on Twitter at
<li> <a href="/2016/11/15/weekly-update.html">Apache Kudu Weekly Update November 15th, 2016</a> </li>
- <li> <a href="/2016/11/01/weekly-update.html">Apache Kudu Weekly Update November 1st, 2016</a> </li>
-
</ul>
</div>
</div>
http://git-wip-us.apache.org/repos/asf/kudu-site/blob/854be1d3/2016/09/16/predicate-pushdown.html
----------------------------------------------------------------------
diff --git a/2016/09/16/predicate-pushdown.html b/2016/09/16/predicate-pushdown.html
index a1c26a2..d17a2c5 100644
--- a/2016/09/16/predicate-pushdown.html
+++ b/2016/09/16/predicate-pushdown.html
@@ -269,6 +269,8 @@ coordinators, and from the Cloudera community as a whole.</p>
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2018/10/26/apache-kudu-1-8-0-released.html">Apache Kudu 1.8.0 Released</a> </li>
+
<li> <a href="/2018/09/26/index-skip-scan-optimization-in-kudu.html">Index Skip Scan Optimization in Kudu</a> </li>
<li> <a href="/2018/09/11/simplified-pipelines-with-kudu.html">Simplified Data Pipelines with Kudu</a> </li>
@@ -297,8 +299,6 @@ coordinators, and from the Cloudera community as a whole.</p>
<li> <a href="/2016/11/15/weekly-update.html">Apache Kudu Weekly Update November 15th, 2016</a> </li>
- <li> <a href="/2016/11/01/weekly-update.html">Apache Kudu Weekly Update November 1st, 2016</a> </li>
-
</ul>
</div>
</div>