You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by gi...@apache.org on 2020/10/06 00:04:36 UTC
[beam] branch asf-site updated: Publishing website 2020/10/06
00:04:05 at commit 82cfa29
This is an automated email from the ASF dual-hosted git repository.
git-site-role pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/asf-site by this push:
new bf3b7d5 Publishing website 2020/10/06 00:04:05 at commit 82cfa29
bf3b7d5 is described below
commit bf3b7d5571fa6157ad0dfa0a445c3a459e431341
Author: jenkins <bu...@apache.org>
AuthorDate: Tue Oct 6 00:04:05 2020 +0000
Publishing website 2020/10/06 00:04:05 at commit 82cfa29
---
website/generated-content/get-started/index.xml | 15 +++++++++------
.../get-started/wordcount-example/index.html | 15 +++++++++------
website/generated-content/sitemap.xml | 2 +-
3 files changed, 19 insertions(+), 13 deletions(-)
diff --git a/website/generated-content/get-started/index.xml b/website/generated-content/get-started/index.xml
index 4d03d81..5f6b241 100644
--- a/website/generated-content/get-started/index.xml
+++ b/website/generated-content/get-started/index.xml
@@ -2777,7 +2777,7 @@ each element in the <code>PCollection</code>.</p>
<div class="highlight"><pre class="chroma"><code class="language-java" data-lang="java"><span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="n">ParDo</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="k">new</span> <span class="n">AddTimestampFn</span><span class="o">(</span><span class="n">minTimestamp</span><span class="o">,</sp [...]
</div>
<div class=language-py>
-<div class="highlight"><pre class="chroma"><code class="language-py" data-lang="py"><span class="n">beam</span><span class="o">.</span><span class="n">Map</span><span class="p">(</span><span class="n">AddTimestampFn</span><span class="p">(</span><span class="n">timestamp_seconds</span><span class="p">))</span></code></pre></div>
+<div class="highlight"><pre class="chroma"><code class="language-py" data-lang="py"><span class="n">beam</span><span class="o">.</span><span class="n">Map</span><span class="p">(</span><span class="n">AddTimestampFn</span><span class="p">(</span><span class="n">min_timestamp</span><span class="p">,</span> <span class="n">max_timestamp</span><span class="p">))</span></code></pre></div>
</div>
<div class=language-go>
<div class="highlight"><pre class="chroma"><code class="language-go" data-lang="go"><span class="nx">timestampedLines</span> <span class="o">:=</span> <span class="nx">beam</span><span class="p">.</span><span class="nf">ParDo</span><span class="p">(</span><span class="nx">s</span><span class="p">,</span> <span class="o">&amp;</span><span class="nx">addTimestampFn</span><span class="p">{</span><span class="n [...]
@@ -2925,17 +2925,18 @@ python -m apache_beam.examples.streaming_wordcount \
<h3 id="reading-an-unbounded-dataset">Reading an unbounded dataset</h3>
<p>This example uses an unbounded dataset as input. The code reads Pub/Sub
messages from a Pub/Sub subscription or topic using
-<a href="https://beam.apache.org/releases/pydoc/2.24.0/apache_beam.io.gcp.pubsub.html#apache_beam.io.gcp.pubsub.ReadStringsFromPubSub"><code>beam.io.ReadStringsFromPubSub</code></a>.</p>
+<a href="https://beam.apache.org/releases/pydoc/2.24.0/apache_beam.io.gcp.pubsub.html#apache_beam.io.gcp.pubsub.ReadFromPubSub"><code>beam.io.ReadFromPubSub</code></a>.</p>
<div class=language-java>
<div class="highlight"><pre class="chroma"><code class="language-java" data-lang="java"> <span class="o">//</span> <span class="n">This</span> <span class="n">example</span> <span class="n">is</span> <span class="n">not</span> <span class="n">currently</span> <span class="n">available</span> <span class="k">for</span> <span class="n">the</span> <span class="n">Beam</span> <span class="n">SDK</span> <span class= [...]
</div>
<div class=language-py>
<div class="highlight"><pre class="chroma"><code class="language-py" data-lang="py"> <span class="c1"># Read from Pub/Sub into a PCollection.</span>
<span class="k">if</span> <span class="n">known_args</span><span class="o">.</span><span class="n">input_subscription</span><span class="p">:</span>
-<span class="n">lines</span> <span class="o">=</span> <span class="n">p</span> <span class="o">|</span> <span class="n">beam</span><span class="o">.</span><span class="n">io</span><span class="o">.</span><span class="n">ReadStringsFromPubSub</span><span class="p">(</span>
+<span class="n">data</span> <span class="o">=</span> <span class="n">p</span> <span class="o">|</span> <span class="n">beam</span><span class="o">.</span><span class="n">io</span><span class="o">.</span><span class="n">ReadFromPubSub</span><span class="p">(</span>
<span class="n">subscription</span><span class="o">=</span><span class="n">known_args</span><span class="o">.</span><span class="n">input_subscription</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
-<span class="n">lines</span> <span class="o">=</span> <span class="n">p</span> <span class="o">|</span> <span class="n">beam</span><span class="o">.</span><span class="n">io</span><span class="o">.</span><span class="n">ReadStringsFromPubSub</span><span class="p">(</span><span class="n">topic</span><span class="o">=</span><span class="n">known_args</span><span class="o">.</span><span class="n">input_to [...]
+<span class="n">data</span> <span class="o">=</span> <span class="n">p</span> <span class="o">|</span> <span class="n">beam</span><span class="o">.</span><span class="n">io</span><span class="o">.</span><span class="n">ReadFromPubSub</span><span class="p">(</span><span class="n">topic</span><span class="o">=</span><span class="n">known_args</span><span class="o">.</span><span class="n">input_topic</ [...]
+<span class="n">lines</span> <span class="o">=</span> <span class="n">data</span> <span class="o">|</span> <span class="s1">&#39;DecodeString&#39;</span> <span class="o">&gt;&gt;</span> <span class="n">beam</span><span class="o">.</span><span class="n">Map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">d</span><span class="p">:</span> <span class="n">d</span>&l [...]
</div>
<div class=language-go>
<div class="highlight"><pre class="chroma"><code class="language-go" data-lang="go"> <span class="o">//</span> <span class="nx">This</span> <span class="nx">example</span> <span class="nx">is</span> <span class="nx">not</span> <span class="nx">currently</span> <span class="nx">available</span> <span class="k">for</span> <span class="nx">the</span> <span class="nx">Beam</span> <span class="nx">SDK</span> <span c [...]
@@ -2947,13 +2948,15 @@ support only bounded output, while others support both bounded and unbounded
outputs.</p>
<p>This example uses an unbounded <code>PCollection</code> and streams the results to
Google Pub/Sub. The code formats the results and writes them to a Pub/Sub topic
-using <a href="https://beam.apache.org/releases/pydoc/2.24.0/apache_beam.io.gcp.pubsub.html#apache_beam.io.gcp.pubsub.WriteStringsToPubSub"><code>beam.io.WriteStringsToPubSub</code></a>.</p>
+using <a href="https://beam.apache.org/releases/pydoc/2.24.0/apache_beam.io.gcp.pubsub.html#apache_beam.io.gcp.pubsub.WriteToPubSub"><code>beam.io.WriteToPubSub</code></a>.</p>
<div class=language-java>
<div class="highlight"><pre class="chroma"><code class="language-java" data-lang="java"> <span class="o">//</span> <span class="n">This</span> <span class="n">example</span> <span class="n">is</span> <span class="n">not</span> <span class="n">currently</span> <span class="n">available</span> <span class="k">for</span> <span class="n">the</span> <span class="n">Beam</span> <span class="n">SDK</span> <span class= [...]
</div>
<div class=language-py>
<div class="highlight"><pre class="chroma"><code class="language-py" data-lang="py"> <span class="c1"># Write to Pub/Sub</span>
-<span class="n">output</span> <span class="o">|</span> <span class="n">beam</span><span class="o">.</span><span class="n">io</span><span class="o">.</span><span class="n">WriteStringsToPubSub</span><span class="p">(</span><span class="n">known_args</span><span class="o">.</span><span class="n">output_topic</span><span class="p">)</span></code></pre></div>
+<span class="n">_</span> <span class="o">=</span> <span class="p">(</span><span class="n">output</span>
+<span class="o">|</span> <span class="s1">&#39;EncodeString&#39;</span> <span class="o">&gt;&gt;</span> <span class="n">Map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">s</span><span class="p">:</span> <span class="n">s</span><span class="o">.</span><span class="n">encode</span><span class="p">(</span><span class="s1">&#39;utf-8&#39;</span><span class="p [...]
+<span class="o">|</span> <span class="n">beam</span><span class="o">.</span><span class="n">io</span><span class="o">.</span><span class="n">WriteToPubSub</span><span class="p">(</span><span class="n">known_args</span><span class="o">.</span><span class="n">output_topic</span><span class="p">))</span></code></pre></div>
</div>
<div class=language-go>
<div class="highlight"><pre class="chroma"><code class="language-go" data-lang="go"> <span class="o">//</span> <span class="nx">This</span> <span class="nx">example</span> <span class="nx">is</span> <span class="nx">not</span> <span class="nx">currently</span> <span class="nx">available</span> <span class="k">for</span> <span class="nx">the</span> <span class="nx">Beam</span> <span class="nx">SDK</span> <span c [...]
diff --git a/website/generated-content/get-started/wordcount-example/index.html b/website/generated-content/get-started/wordcount-example/index.html
index c38d600..6595094 100644
--- a/website/generated-content/get-started/wordcount-example/index.html
+++ b/website/generated-content/get-started/wordcount-example/index.html
@@ -490,7 +490,7 @@ each new element a timestamp that corresponds to when the element was read or
added. You can manually assign or adjust timestamps with a <code>DoFn</code>; however, you
can only move timestamps forward in time.</p><p>In this example the input is bounded. For the purpose of the example, the <code>DoFn</code>
method named <code>AddTimestampsFn</code> (invoked by <code>ParDo</code>) will set a timestamp for
-each element in the <code>PCollection</code>.</p><div class=language-java><div class=highlight><pre class=chroma><code class=language-java data-lang=java><span class=o>.</span><span class=na>apply</span><span class=o>(</span><span class=n>ParDo</span><span class=o>.</span><span class=na>of</span><span class=o>(</span><span class=k>new</span> <span class=n>AddTimestampFn</span><span class=o>(</span><span class=n>minTimestamp</span><span class=o>,</span> <span class=n>maxTimestamp</span><s [...]
+each element in the <code>PCollection</code>.</p><div class=language-java><div class=highlight><pre class=chroma><code class=language-java data-lang=java><span class=o>.</span><span class=na>apply</span><span class=o>(</span><span class=n>ParDo</span><span class=o>.</span><span class=na>of</span><span class=o>(</span><span class=k>new</span> <span class=n>AddTimestampFn</span><span class=o>(</span><span class=n>minTimestamp</span><span class=o>,</span> <span class=n>maxTimestamp</span><s [...]
the data element of the timestamp given the element itself. For example, if the
elements were log lines, this <code>ParDo</code> could parse the time out of the log string
and set it as the element’s timestamp. There are no timestamps inherent in the
@@ -563,18 +563,21 @@ python -m apache_beam.examples.streaming_wordcount \
<strong><a href=https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/streaming_wordcount.py>streaming_wordcount.py</a>.</strong></p><p><strong>To run this example in Go:</strong></p><blockquote><p><strong>Note:</strong> StreamingWordCount is not yet available for the Go SDK. There is an open issue for this
(<a href=https://issues.apache.org/jira/browse/BEAM-4292>BEAM-4292</a>).</p></blockquote><h3 id=reading-an-unbounded-dataset>Reading an unbounded dataset</h3><p>This example uses an unbounded dataset as input. The code reads Pub/Sub
messages from a Pub/Sub subscription or topic using
-<a href=https://beam.apache.org/releases/pydoc/2.24.0/apache_beam.io.gcp.pubsub.html#apache_beam.io.gcp.pubsub.ReadStringsFromPubSub><code>beam.io.ReadStringsFromPubSub</code></a>.</p><div class=language-java><div class=highlight><pre class=chroma><code class=language-java data-lang=java> <span class=o>//</span> <span class=n>This</span> <span class=n>example</span> <span class=n>is</span> <span class=n>not</span> <span class=n>currently</span> <span class=n>available</span> <span class [...]
+<a href=https://beam.apache.org/releases/pydoc/2.24.0/apache_beam.io.gcp.pubsub.html#apache_beam.io.gcp.pubsub.ReadFromPubSub><code>beam.io.ReadFromPubSub</code></a>.</p><div class=language-java><div class=highlight><pre class=chroma><code class=language-java data-lang=java> <span class=o>//</span> <span class=n>This</span> <span class=n>example</span> <span class=n>is</span> <span class=n>not</span> <span class=n>currently</span> <span class=n>available</span> <span class=k>for</span> [...]
<span class=k>if</span> <span class=n>known_args</span><span class=o>.</span><span class=n>input_subscription</span><span class=p>:</span>
- <span class=n>lines</span> <span class=o>=</span> <span class=n>p</span> <span class=o>|</span> <span class=n>beam</span><span class=o>.</span><span class=n>io</span><span class=o>.</span><span class=n>ReadStringsFromPubSub</span><span class=p>(</span>
+ <span class=n>data</span> <span class=o>=</span> <span class=n>p</span> <span class=o>|</span> <span class=n>beam</span><span class=o>.</span><span class=n>io</span><span class=o>.</span><span class=n>ReadFromPubSub</span><span class=p>(</span>
<span class=n>subscription</span><span class=o>=</span><span class=n>known_args</span><span class=o>.</span><span class=n>input_subscription</span><span class=p>)</span>
<span class=k>else</span><span class=p>:</span>
- <span class=n>lines</span> <span class=o>=</span> <span class=n>p</span> <span class=o>|</span> <span class=n>beam</span><span class=o>.</span><span class=n>io</span><span class=o>.</span><span class=n>ReadStringsFromPubSub</span><span class=p>(</span><span class=n>topic</span><span class=o>=</span><span class=n>known_args</span><span class=o>.</span><span class=n>input_topic</span><span class=p>)</span></code></pre></div></div><div class=language-go><div class=highlight><pre class=c [...]
+ <span class=n>data</span> <span class=o>=</span> <span class=n>p</span> <span class=o>|</span> <span class=n>beam</span><span class=o>.</span><span class=n>io</span><span class=o>.</span><span class=n>ReadFromPubSub</span><span class=p>(</span><span class=n>topic</span><span class=o>=</span><span class=n>known_args</span><span class=o>.</span><span class=n>input_topic</span><span class=p>)</span>
+ <span class=n>lines</span> <span class=o>=</span> <span class=n>data</span> <span class=o>|</span> <span class=s1>'DecodeString'</span> <span class=o>>></span> <span class=n>beam</span><span class=o>.</span><span class=n>Map</span><span class=p>(</span><span class=k>lambda</span> <span class=n>d</span><span class=p>:</span> <span class=n>d</span><span class=o>.</span><span class=n>decode</span><span class=p>(</span><span class=s1>'utf-8'</span><span class=p>))</sp [...]
such, you must make sure to choose an appropriate I/O for the results. Some I/Os
support only bounded output, while others support both bounded and unbounded
outputs.</p><p>This example uses an unbounded <code>PCollection</code> and streams the results to
Google Pub/Sub. The code formats the results and writes them to a Pub/Sub topic
-using <a href=https://beam.apache.org/releases/pydoc/2.24.0/apache_beam.io.gcp.pubsub.html#apache_beam.io.gcp.pubsub.WriteStringsToPubSub><code>beam.io.WriteStringsToPubSub</code></a>.</p><div class=language-java><div class=highlight><pre class=chroma><code class=language-java data-lang=java> <span class=o>//</span> <span class=n>This</span> <span class=n>example</span> <span class=n>is</span> <span class=n>not</span> <span class=n>currently</span> <span class=n>available</span> <span c [...]
- <span class=n>output</span> <span class=o>|</span> <span class=n>beam</span><span class=o>.</span><span class=n>io</span><span class=o>.</span><span class=n>WriteStringsToPubSub</span><span class=p>(</span><span class=n>known_args</span><span class=o>.</span><span class=n>output_topic</span><span class=p>)</span></code></pre></div></div><div class=language-go><div class=highlight><pre class=chroma><code class=language-go data-lang=go> <span class=o>//</span> <span class=nx>This</span> [...]
+using <a href=https://beam.apache.org/releases/pydoc/2.24.0/apache_beam.io.gcp.pubsub.html#apache_beam.io.gcp.pubsub.WriteToPubSub><code>beam.io.WriteToPubSub</code></a>.</p><div class=language-java><div class=highlight><pre class=chroma><code class=language-java data-lang=java> <span class=o>//</span> <span class=n>This</span> <span class=n>example</span> <span class=n>is</span> <span class=n>not</span> <span class=n>currently</span> <span class=n>available</span> <span class=k>for</sp [...]
+ <span class=n>_</span> <span class=o>=</span> <span class=p>(</span><span class=n>output</span>
+ <span class=o>|</span> <span class=s1>'EncodeString'</span> <span class=o>>></span> <span class=n>Map</span><span class=p>(</span><span class=k>lambda</span> <span class=n>s</span><span class=p>:</span> <span class=n>s</span><span class=o>.</span><span class=n>encode</span><span class=p>(</span><span class=s1>'utf-8'</span><span class=p>))</span>
+ <span class=o>|</span> <span class=n>beam</span><span class=o>.</span><span class=n>io</span><span class=o>.</span><span class=n>WriteToPubSub</span><span class=p>(</span><span class=n>known_args</span><span class=o>.</span><span class=n>output_topic</span><span class=p>))</span></code></pre></div></div><div class=language-go><div class=highlight><pre class=chroma><code class=language-go data-lang=go> <span class=o>//</span> <span class=nx>This</span> <span class=nx>example</span> < [...]
<a href=http://www.apache.org>The Apache Software Foundation</a>
| <a href=/privacy_policy>Privacy Policy</a>
| <a href=/feed.xml>RSS Feed</a><br><br>Apache Beam, Apache, Beam, the Beam logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation. All other products or name brands are trademarks of their respective holders, including The Apache Software Foundation.</div></footer></body></html>
\ No newline at end of file
diff --git a/website/generated-content/sitemap.xml b/website/generated-content/sitemap.xml
index 8b5b93f..16088ee 100644
--- a/website/generated-content/sitemap.xml
+++ b/website/generated-content/sitemap.xml
@@ -1 +1 @@
-<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml"><url><loc>/blog/beam-2.24.0/</loc><lastmod>2020-09-18T12:38:38-07:00</lastmod></url><url><loc>/categories/blog/</loc><lastmod>2020-09-18T12:38:38-07:00</lastmod></url><url><loc>/blog/</loc><lastmod>2020-09-18T12:38:38-07:00</lastmod></url><url><loc>/categories/</loc><lastmod>2020-09-18T12:38:38-07:00</lastmod></url><url><loc>/blog/p [...]
\ No newline at end of file
+<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml"><url><loc>/blog/beam-2.24.0/</loc><lastmod>2020-09-18T12:38:38-07:00</lastmod></url><url><loc>/categories/blog/</loc><lastmod>2020-09-18T12:38:38-07:00</lastmod></url><url><loc>/blog/</loc><lastmod>2020-09-18T12:38:38-07:00</lastmod></url><url><loc>/categories/</loc><lastmod>2020-09-18T12:38:38-07:00</lastmod></url><url><loc>/blog/p [...]
\ No newline at end of file