You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by yh...@apache.org on 2016/12/28 22:35:14 UTC
[03/25] spark-website git commit: Update 2.1.0 docs to include
https://github.com/apache/spark/pull/16294
http://git-wip-us.apache.org/repos/asf/spark-website/blob/d2bcf185/site/docs/2.1.0/structured-streaming-kafka-integration.html
----------------------------------------------------------------------
diff --git a/site/docs/2.1.0/structured-streaming-kafka-integration.html b/site/docs/2.1.0/structured-streaming-kafka-integration.html
index 5ca9259..7d2254f 100644
--- a/site/docs/2.1.0/structured-streaming-kafka-integration.html
+++ b/site/docs/2.1.0/structured-streaming-kafka-integration.html
@@ -144,7 +144,7 @@ application. See the <a href="#deploying">Deploying</a> subsection below.</p>
<div class="codetabs">
<div data-lang="scala">
- <div class="highlight"><pre><code class="language-scala" data-lang="scala"><span class="c1">// Subscribe to 1 topic</span>
+ <figure class="highlight"><pre><code class="language-scala" data-lang="scala"><span></span><span class="c1">// Subscribe to 1 topic</span>
<span class="k">val</span> <span class="n">ds1</span> <span class="k">=</span> <span class="n">spark</span>
<span class="o">.</span><span class="n">readStream</span>
<span class="o">.</span><span class="n">format</span><span class="o">(</span><span class="s">"kafka"</span><span class="o">)</span>
@@ -172,12 +172,12 @@ application. See the <a href="#deploying">Deploying</a> subsection below.</p>
<span class="o">.</span><span class="n">option</span><span class="o">(</span><span class="s">"subscribePattern"</span><span class="o">,</span> <span class="s">"topic.*"</span><span class="o">)</span>
<span class="o">.</span><span class="n">load</span><span class="o">()</span>
<span class="n">ds3</span><span class="o">.</span><span class="n">selectExpr</span><span class="o">(</span><span class="s">"CAST(key AS STRING)"</span><span class="o">,</span> <span class="s">"CAST(value AS STRING)"</span><span class="o">)</span>
- <span class="o">.</span><span class="n">as</span><span class="o">[(</span><span class="kt">String</span>, <span class="kt">String</span><span class="o">)]</span></code></pre></div>
+ <span class="o">.</span><span class="n">as</span><span class="o">[(</span><span class="kt">String</span>, <span class="kt">String</span><span class="o">)]</span></code></pre></figure>
</div>
<div data-lang="java">
- <div class="highlight"><pre><code class="language-java" data-lang="java"><span class="c1">// Subscribe to 1 topic</span>
+ <figure class="highlight"><pre><code class="language-java" data-lang="java"><span></span><span class="c1">// Subscribe to 1 topic</span>
<span class="n">Dataset</span><span class="o"><</span><span class="n">Row</span><span class="o">></span> <span class="n">ds1</span> <span class="o">=</span> <span class="n">spark</span>
<span class="o">.</span><span class="na">readStream</span><span class="o">()</span>
<span class="o">.</span><span class="na">format</span><span class="o">(</span><span class="s">"kafka"</span><span class="o">)</span>
@@ -202,43 +202,43 @@ application. See the <a href="#deploying">Deploying</a> subsection below.</p>
<span class="o">.</span><span class="na">option</span><span class="o">(</span><span class="s">"kafka.bootstrap.servers"</span><span class="o">,</span> <span class="s">"host1:port1,host2:port2"</span><span class="o">)</span>
<span class="o">.</span><span class="na">option</span><span class="o">(</span><span class="s">"subscribePattern"</span><span class="o">,</span> <span class="s">"topic.*"</span><span class="o">)</span>
<span class="o">.</span><span class="na">load</span><span class="o">()</span>
-<span class="n">ds3</span><span class="o">.</span><span class="na">selectExpr</span><span class="o">(</span><span class="s">"CAST(key AS STRING)"</span><span class="o">,</span> <span class="s">"CAST(value AS STRING)"</span><span class="o">)</span></code></pre></div>
+<span class="n">ds3</span><span class="o">.</span><span class="na">selectExpr</span><span class="o">(</span><span class="s">"CAST(key AS STRING)"</span><span class="o">,</span> <span class="s">"CAST(value AS STRING)"</span><span class="o">)</span></code></pre></figure>
</div>
<div data-lang="python">
- <div class="highlight"><pre><code class="language-python" data-lang="python"><span class="c"># Subscribe to 1 topic</span>
+ <figure class="highlight"><pre><code class="language-python" data-lang="python"><span></span><span class="c1"># Subscribe to 1 topic</span>
<span class="n">ds1</span> <span class="o">=</span> <span class="n">spark</span>
<span class="o">.</span><span class="n">readStream</span><span class="p">()</span>
- <span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="s">"kafka"</span><span class="p">)</span>
- <span class="o">.</span><span class="n">option</span><span class="p">(</span><span class="s">"kafka.bootstrap.servers"</span><span class="p">,</span> <span class="s">"host1:port1,host2:port2"</span><span class="p">)</span>
- <span class="o">.</span><span class="n">option</span><span class="p">(</span><span class="s">"subscribe"</span><span class="p">,</span> <span class="s">"topic1"</span><span class="p">)</span>
+ <span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="s2">"kafka"</span><span class="p">)</span>
+ <span class="o">.</span><span class="n">option</span><span class="p">(</span><span class="s2">"kafka.bootstrap.servers"</span><span class="p">,</span> <span class="s2">"host1:port1,host2:port2"</span><span class="p">)</span>
+ <span class="o">.</span><span class="n">option</span><span class="p">(</span><span class="s2">"subscribe"</span><span class="p">,</span> <span class="s2">"topic1"</span><span class="p">)</span>
<span class="o">.</span><span class="n">load</span><span class="p">()</span>
-<span class="n">ds1</span><span class="o">.</span><span class="n">selectExpr</span><span class="p">(</span><span class="s">"CAST(key AS STRING)"</span><span class="p">,</span> <span class="s">"CAST(value AS STRING)"</span><span class="p">)</span>
+<span class="n">ds1</span><span class="o">.</span><span class="n">selectExpr</span><span class="p">(</span><span class="s2">"CAST(key AS STRING)"</span><span class="p">,</span> <span class="s2">"CAST(value AS STRING)"</span><span class="p">)</span>
-<span class="c"># Subscribe to multiple topics</span>
+<span class="c1"># Subscribe to multiple topics</span>
<span class="n">ds2</span> <span class="o">=</span> <span class="n">spark</span>
<span class="o">.</span><span class="n">readStream</span>
- <span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="s">"kafka"</span><span class="p">)</span>
- <span class="o">.</span><span class="n">option</span><span class="p">(</span><span class="s">"kafka.bootstrap.servers"</span><span class="p">,</span> <span class="s">"host1:port1,host2:port2"</span><span class="p">)</span>
- <span class="o">.</span><span class="n">option</span><span class="p">(</span><span class="s">"subscribe"</span><span class="p">,</span> <span class="s">"topic1,topic2"</span><span class="p">)</span>
+ <span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="s2">"kafka"</span><span class="p">)</span>
+ <span class="o">.</span><span class="n">option</span><span class="p">(</span><span class="s2">"kafka.bootstrap.servers"</span><span class="p">,</span> <span class="s2">"host1:port1,host2:port2"</span><span class="p">)</span>
+ <span class="o">.</span><span class="n">option</span><span class="p">(</span><span class="s2">"subscribe"</span><span class="p">,</span> <span class="s2">"topic1,topic2"</span><span class="p">)</span>
<span class="o">.</span><span class="n">load</span><span class="p">()</span>
-<span class="n">ds2</span><span class="o">.</span><span class="n">selectExpr</span><span class="p">(</span><span class="s">"CAST(key AS STRING)"</span><span class="p">,</span> <span class="s">"CAST(value AS STRING)"</span><span class="p">)</span>
+<span class="n">ds2</span><span class="o">.</span><span class="n">selectExpr</span><span class="p">(</span><span class="s2">"CAST(key AS STRING)"</span><span class="p">,</span> <span class="s2">"CAST(value AS STRING)"</span><span class="p">)</span>
-<span class="c"># Subscribe to a pattern</span>
+<span class="c1"># Subscribe to a pattern</span>
<span class="n">ds3</span> <span class="o">=</span> <span class="n">spark</span>
<span class="o">.</span><span class="n">readStream</span><span class="p">()</span>
- <span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="s">"kafka"</span><span class="p">)</span>
- <span class="o">.</span><span class="n">option</span><span class="p">(</span><span class="s">"kafka.bootstrap.servers"</span><span class="p">,</span> <span class="s">"host1:port1,host2:port2"</span><span class="p">)</span>
- <span class="o">.</span><span class="n">option</span><span class="p">(</span><span class="s">"subscribePattern"</span><span class="p">,</span> <span class="s">"topic.*"</span><span class="p">)</span>
+ <span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="s2">"kafka"</span><span class="p">)</span>
+ <span class="o">.</span><span class="n">option</span><span class="p">(</span><span class="s2">"kafka.bootstrap.servers"</span><span class="p">,</span> <span class="s2">"host1:port1,host2:port2"</span><span class="p">)</span>
+ <span class="o">.</span><span class="n">option</span><span class="p">(</span><span class="s2">"subscribePattern"</span><span class="p">,</span> <span class="s2">"topic.*"</span><span class="p">)</span>
<span class="o">.</span><span class="n">load</span><span class="p">()</span>
-<span class="n">ds3</span><span class="o">.</span><span class="n">selectExpr</span><span class="p">(</span><span class="s">"CAST(key AS STRING)"</span><span class="p">,</span> <span class="s">"CAST(value AS STRING)"</span><span class="p">)</span></code></pre></div>
+<span class="n">ds3</span><span class="o">.</span><span class="n">selectExpr</span><span class="p">(</span><span class="s2">"CAST(key AS STRING)"</span><span class="p">,</span> <span class="s2">"CAST(value AS STRING)"</span><span class="p">)</span></code></pre></figure>
</div>
</div>
-<p>Each row in the source has the following schema:</p>
-<table class="table">
+<p>Each row in the source has the following schema:
+<table class="table"></p>
<tr><th>Column</th><th>Type</th></tr>
<tr>
<td>key</td>
@@ -268,7 +268,7 @@ application. See the <a href="#deploying">Deploying</a> subsection below.</p>
<td>timestampType</td>
<td>int</td>
</tr>
-</table>
+<p></table></p>
<p>The following options must be set for the Kafka source.</p>
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org