You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by gi...@apache.org on 2022/07/20 04:18:26 UTC
[beam] branch asf-site updated: Publishing website 2022/07/20 04:18:18 at commit ba62704
This is an automated email from the ASF dual-hosted git repository.
git-site-role pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 6284fadbd58 Publishing website 2022/07/20 04:18:18 at commit ba62704
6284fadbd58 is described below
commit 6284fadbd58d9569ad6f0bc70b79e1d2a44a754a
Author: jenkins <bu...@apache.org>
AuthorDate: Wed Jul 20 04:18:18 2022 +0000
Publishing website 2022/07/20 04:18:18 at commit ba62704
---
website/generated-content/documentation/index.xml | 20 ++++++++++++++------
.../documentation/programming-guide/index.html | 20 ++++++++++++++------
2 files changed, 28 insertions(+), 12 deletions(-)
diff --git a/website/generated-content/documentation/index.xml b/website/generated-content/documentation/index.xml
index 5c1862f31ac..b9582243350 100644
--- a/website/generated-content/documentation/index.xml
+++ b/website/generated-content/documentation/index.xml
@@ -5800,8 +5800,12 @@ data contains names and phone numbers.</p>
<span class="p">{</span> <span class="nx">name</span><span class="o">:</span> <span class="s2">&#34;amy&#34;</span><span class="p">,</span> <span class="nx">phone</span><span class="o">:</span> <span class="s2">&#34;333-444-5555&#34;</span> <span class="p">},</span>
<span class="p">{</span> <span class="nx">name</span><span class="o">:</span> <span class="s2">&#34;carl&#34;</span><span class="p">,</span> <span class="nx">phone</span><span class="o">:</span> <span class="s2">&#34;444-555-6666&#34;</span> <span class="p">},</span>
<span class="p">];</span>
-<span class="kr">const</span> <span class="nx">emails</span> <span class="o">=</span> <span class="nx">root</span><span class="p">.</span><span class="nx">apply</span><span class="p">(</span><span class="nx">beam</span><span class="p">.</span><span class="nx">create</span><span class="p">(</span><span class="nx">emails_list</span><span class="p">));</span>
-<span class="kr">const</span> <span class="nx">phones</span> <span class="o">=</span> <span class="nx">root</span><span class="p">.</span><span class="nx">apply</span><span class="p">(</span><span class="nx">beam</span><span class="p">.</span><span class="nx">create</span><span class="p">(</span><span class="nx">phones_list</span><span class="p">));</span>
+<span class="kr">const</span> <span class="nx">emails</span> <span class="o">=</span> <span class="nx">root</span><span class="p">.</span><span class="nx">apply</span><span class="p">(</span>
+<span class="nx">beam</span><span class="p">.</span><span class="nx">withName</span><span class="p">(</span><span class="s2">&#34;createEmails&#34;</span><span class="p">,</span> <span class="nx">beam</span><span class="p">.</span><span class="nx">create</span><span class="p">(</span><span class="nx">emails_list</span><span class="p">))</span>
+<span class="p">);</span>
+<span class="kr">const</span> <span class="nx">phones</span> <span class="o">=</span> <span class="nx">root</span><span class="p">.</span><span class="nx">apply</span><span class="p">(</span>
+<span class="nx">beam</span><span class="p">.</span><span class="nx">withName</span><span class="p">(</span><span class="s2">&#34;createPhones&#34;</span><span class="p">,</span> <span class="nx">beam</span><span class="p">.</span><span class="nx">create</span><span class="p">(</span><span class="nx">phones_list</span><span class="p">))</span>
+<span class="p">);</span>
</code></pre></div>
</div>
</div>
@@ -6653,8 +6657,12 @@ is a Beam transform for <code>PCollection</code> objects that store the sa
<div class="highlight"><pre class="chroma"><code class="language-typescript" data-lang="typescript"><span class="c1">// Flatten takem an array of PCollection objects, wrapped in beam.P(...)
</span><span class="c1">// Returns a single PCollection that contains a union of all of the elements in all input PCollections.
</span><span class="c1"></span>
-<span class="kr">const</span> <span class="nx">fib</span> <span class="o">=</span> <span class="nx">root</span><span class="p">.</span><span class="nx">apply</span><span class="p">(</span><span class="nx">beam</span><span class="p">.</span><span class="nx">create</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <spa [...]
-<span class="kr">const</span> <span class="nx">pow</span> <span class="o">=</span> <span class="nx">root</span><span class="p">.</span><span class="nx">apply</span><span class="p">(</span><span class="nx">beam</span><span class="p">.</span><span class="nx">create</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <spa [...]
+<span class="kr">const</span> <span class="nx">fib</span> <span class="o">=</span> <span class="nx">root</span><span class="p">.</span><span class="nx">apply</span><span class="p">(</span>
+<span class="nx">beam</span><span class="p">.</span><span class="nx">withName</span><span class="p">(</span><span class="s2">&#34;createFib&#34;</span><span class="p">,</span> <span class="nx">beam</span><span class="p">.</span><span class="nx">create</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="m [...]
+<span class="p">);</span>
+<span class="kr">const</span> <span class="nx">pow</span> <span class="o">=</span> <span class="nx">root</span><span class="p">.</span><span class="nx">apply</span><span class="p">(</span>
+<span class="nx">beam</span><span class="p">.</span><span class="nx">withName</span><span class="p">(</span><span class="s2">&#34;createPow&#34;</span><span class="p">,</span> <span class="nx">beam</span><span class="p">.</span><span class="nx">create</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="m [...]
+<span class="p">);</span>
<span class="kr">const</span> <span class="nx">result</span> <span class="o">=</span> <span class="nx">beam</span><span class="p">.</span><span class="nx">P</span><span class="p">([</span><span class="nx">fib</span><span class="p">,</span> <span class="nx">pow</span><span class="p">]).</span><span class="nx">apply</span><span class="p">(</span><span class="nx">beam</span><span class="p">.</span>< [...]
</code></pre></div>
</div>
@@ -7437,7 +7445,7 @@ function will be invoked multiple time for the element, once for each window.<
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
<img src="/images/copy-icon.svg"/>
</a>
-<div class="highlight"><pre class="chroma"><code class="language-typescript" data-lang="typescript"><span class="nx">pcoll</span><span class="p">.</span><span class="nx">map</span><span class="p">(</span><span class="nx">processFn</span><span class="p">,</span> <span class="p">{</span> <span class="nx">timestamp</span>: <span class="kt">pardo.windowParam</span><span class="p">()</span> <span class="p">});</span>
+<div class="highlight"><pre class="chroma"><code class="language-typescript" data-lang="typescript"><span class="nx">pcoll</span><span class="p">.</span><span class="nx">map</span><span class="p">(</span><span class="nx">processWithWindow</span><span class="p">,</span> <span class="p">{</span> <span class="nx">timestamp</span>: <span class="kt">pardo.windowParam</span><span class="p">()</span> <span class="p">});</span>
</code></pre></div>
</div>
</div>
@@ -7507,7 +7515,7 @@ and how many times this window has already fired for this key.</p>
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
<img src="/images/copy-icon.svg"/>
</a>
-<div class="highlight"><pre class="chroma"><code class="language-typescript" data-lang="typescript"><span class="nx">pcoll</span><span class="p">.</span><span class="nx">map</span><span class="p">(</span><span class="nx">processFn</span><span class="p">,</span> <span class="p">{</span> <span class="nx">timestamp</span>: <span class="kt">pardo.paneInfoParam</span><span class="p">()</span> <span class="p">});</span>
+<div class="highlight"><pre class="chroma"><code class="language-typescript" data-lang="typescript"><span class="nx">pcoll</span><span class="p">.</span><span class="nx">map</span><span class="p">(</span><span class="nx">processWithPaneInfo</span><span class="p">,</span> <span class="p">{</span> <span class="nx">timestamp</span>: <span class="kt">pardo.paneInfoParam</span><span class="p">()</span> <span class="p">}); [...]
</code></pre></div>
</div>
</div>
diff --git a/website/generated-content/documentation/programming-guide/index.html b/website/generated-content/documentation/programming-guide/index.html
index 00769b0874d..c755000bbda 100644
--- a/website/generated-content/documentation/programming-guide/index.html
+++ b/website/generated-content/documentation/programming-guide/index.html
@@ -909,8 +909,12 @@ data contains names and phone numbers.</p><div class="language-java snippet"><di
<span class=p>{</span> <span class=nx>name</span><span class=o>:</span> <span class=s2>"carl"</span><span class=p>,</span> <span class=nx>phone</span><span class=o>:</span> <span class=s2>"444-555-6666"</span> <span class=p>},</span>
<span class=p>];</span>
-<span class=kr>const</span> <span class=nx>emails</span> <span class=o>=</span> <span class=nx>root</span><span class=p>.</span><span class=nx>apply</span><span class=p>(</span><span class=nx>beam</span><span class=p>.</span><span class=nx>create</span><span class=p>(</span><span class=nx>emails_list</span><span class=p>));</span>
-<span class=kr>const</span> <span class=nx>phones</span> <span class=o>=</span> <span class=nx>root</span><span class=p>.</span><span class=nx>apply</span><span class=p>(</span><span class=nx>beam</span><span class=p>.</span><span class=nx>create</span><span class=p>(</span><span class=nx>phones_list</span><span class=p>));</span>
+<span class=kr>const</span> <span class=nx>emails</span> <span class=o>=</span> <span class=nx>root</span><span class=p>.</span><span class=nx>apply</span><span class=p>(</span>
+ <span class=nx>beam</span><span class=p>.</span><span class=nx>withName</span><span class=p>(</span><span class=s2>"createEmails"</span><span class=p>,</span> <span class=nx>beam</span><span class=p>.</span><span class=nx>create</span><span class=p>(</span><span class=nx>emails_list</span><span class=p>))</span>
+<span class=p>);</span>
+<span class=kr>const</span> <span class=nx>phones</span> <span class=o>=</span> <span class=nx>root</span><span class=p>.</span><span class=nx>apply</span><span class=p>(</span>
+ <span class=nx>beam</span><span class=p>.</span><span class=nx>withName</span><span class=p>(</span><span class=s2>"createPhones"</span><span class=p>,</span> <span class=nx>beam</span><span class=p>.</span><span class=nx>create</span><span class=p>(</span><span class=nx>phones_list</span><span class=p>))</span>
+<span class=p>);</span>
</code></pre></div></div></div><p>After <code>CoGroupByKey</code>, the resulting data contains all data associated with each
unique key from any of the input collections.</p><div class="language-java snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><div class=highlight><pre class=chroma><code class=language-java data-lang=java><span class=kd>final</span> <span class=n>TupleTag</span><span class=o><</span><span class=n>String</span><span class=o>></span> <span class=n>emai [...]
<span class=kd>final</span> <span class=n>TupleTag</span><span class=o><</span><span class=n>String</span><span class=o>></span> <span class=n>phonesTag</span> <span class=o>=</span> <span class=k>new</span> <span class=n>TupleTag</span><span class=o><>();</span>
@@ -1457,8 +1461,12 @@ is a Beam transform for <code>PCollection</code> objects that store the same dat
<span class=nx>merged</span> <span class=o>:=</span> <span class=nx>beam</span><span class=p>.</span><span class=nf>Flatten</span><span class=p>(</span><span class=nx>s</span><span class=p>,</span> <span class=nx>pcol1</span><span class=p>,</span> <span class=nx>pcol2</span><span class=p>,</span> <span class=nx>pcol3</span><span class=p>)</span></code></pre></div></div></div><div class="language-typescript snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs [...]
</span><span class=c1>// Returns a single PCollection that contains a union of all of the elements in all input PCollections.
</span><span class=c1></span>
-<span class=kr>const</span> <span class=nx>fib</span> <span class=o>=</span> <span class=nx>root</span><span class=p>.</span><span class=nx>apply</span><span class=p>(</span><span class=nx>beam</span><span class=p>.</span><span class=nx>create</span><span class=p>([</span><span class=mi>1</span><span class=p>,</span> <span class=mi>1</span><span class=p>,</span> <span class=mi>2</span><span class=p>,</span> <span class=mi>3</span><span class=p>,</span> <span class=mi>5</span><span class= [...]
-<span class=kr>const</span> <span class=nx>pow</span> <span class=o>=</span> <span class=nx>root</span><span class=p>.</span><span class=nx>apply</span><span class=p>(</span><span class=nx>beam</span><span class=p>.</span><span class=nx>create</span><span class=p>([</span><span class=mi>1</span><span class=p>,</span> <span class=mi>2</span><span class=p>,</span> <span class=mi>4</span><span class=p>,</span> <span class=mi>8</span><span class=p>,</span> <span class=mi>16</span><span class [...]
+<span class=kr>const</span> <span class=nx>fib</span> <span class=o>=</span> <span class=nx>root</span><span class=p>.</span><span class=nx>apply</span><span class=p>(</span>
+ <span class=nx>beam</span><span class=p>.</span><span class=nx>withName</span><span class=p>(</span><span class=s2>"createFib"</span><span class=p>,</span> <span class=nx>beam</span><span class=p>.</span><span class=nx>create</span><span class=p>([</span><span class=mi>1</span><span class=p>,</span> <span class=mi>1</span><span class=p>,</span> <span class=mi>2</span><span class=p>,</span> <span class=mi>3</span><span class=p>,</span> <span class=mi>5</span><span class=p>,</spa [...]
+<span class=p>);</span>
+<span class=kr>const</span> <span class=nx>pow</span> <span class=o>=</span> <span class=nx>root</span><span class=p>.</span><span class=nx>apply</span><span class=p>(</span>
+ <span class=nx>beam</span><span class=p>.</span><span class=nx>withName</span><span class=p>(</span><span class=s2>"createPow"</span><span class=p>,</span> <span class=nx>beam</span><span class=p>.</span><span class=nx>create</span><span class=p>([</span><span class=mi>1</span><span class=p>,</span> <span class=mi>2</span><span class=p>,</span> <span class=mi>4</span><span class=p>,</span> <span class=mi>8</span><span class=p>,</span> <span class=mi>16</span><span class=p>,</sp [...]
+<span class=p>);</span>
<span class=kr>const</span> <span class=nx>result</span> <span class=o>=</span> <span class=nx>beam</span><span class=p>.</span><span class=nx>P</span><span class=p>([</span><span class=nx>fib</span><span class=p>,</span> <span class=nx>pow</span><span class=p>]).</span><span class=nx>apply</span><span class=p>(</span><span class=nx>beam</span><span class=p>.</span><span class=nx>flatten</span><span class=p>());</span>
</code></pre></div></div></div><h5 id=data-encoding-merged-collections>4.2.5.1. Data encoding in merged collections</h5><p>By default, the coder for the output <code>PCollection</code> is the same as the coder for
the first <code>PCollection</code> in the input <code>PCollectionList</code>. However, the input
@@ -2016,7 +2024,7 @@ function will be invoked multiple time for the element, once for each window.</p
<span class=k>pass</span></code></pre></div></div></div><div class="language-go snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><div class=highlight><pre class=chroma><code class=language-go data-lang=go><span class=kd>func</span> <span class=nf>MyDoFn</span><span class=p>(</span><span class=nx>w</span> <span class=nx>beam</span><span class=p>.</spa [...]
<span class=nx>iw</span> <span class=o>:=</span> <span class=nx>w</span><span class=p>.(</span><span class=nx>window</span><span class=p>.</span><span class=nx>IntervalWindow</span><span class=p>)</span>
<span class=o>...</span>
-<span class=p>}</span></code></pre></div></div></div><div class="language-typescript snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><div class=highlight><pre class=chroma><code class=language-typescript data-lang=typescript><span class=nx>pcoll</span><span class=p>.</span><span class=nx>map</span><span class=p>(</span><span class=nx>processFn</span><spa [...]
+<span class=p>}</span></code></pre></div></div></div><div class="language-typescript snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><div class=highlight><pre class=chroma><code class=language-typescript data-lang=typescript><span class=nx>pcoll</span><span class=p>.</span><span class=nx>map</span><span class=p>(</span><span class=nx>processWithWindow</s [...]
</code></pre></div></div></div><p class=language-java><strong>PaneInfo:</strong>
When triggers are used, Beam provides a <code>PaneInfo</code> object that contains information about the current firing. Using <code>PaneInfo</code>
you can determine whether this is an early or a late firing, and how many times this window has already fired for this key.</p><p class=language-py><strong>PaneInfo:</strong>
@@ -2053,7 +2061,7 @@ and how many times this window has already fired for this key.</p><div class="la
<span class=k>for</span> <span class=nx>_</span><span class=p>,</span> <span class=nx>w</span> <span class=o>:=</span> <span class=k>range</span> <span class=nx>words</span> <span class=p>{</span>
<span class=nf>emitWords</span><span class=p>(</span><span class=nx>w</span><span class=p>)</span>
<span class=p>}</span>
-<span class=p>}</span></code></pre></div></div></div><div class="language-typescript snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><div class=highlight><pre class=chroma><code class=language-typescript data-lang=typescript><span class=nx>pcoll</span><span class=p>.</span><span class=nx>map</span><span class=p>(</span><span class=nx>processFn</span><spa [...]
+<span class=p>}</span></code></pre></div></div></div><div class="language-typescript snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><div class=highlight><pre class=chroma><code class=language-typescript data-lang=typescript><span class=nx>pcoll</span><span class=p>.</span><span class=nx>map</span><span class=p>(</span><span class=nx>processWithPaneInfo< [...]
</code></pre></div></div></div><p class=language-java><strong>PipelineOptions:</strong>
The <code>PipelineOptions</code> for the current pipeline can always be accessed in a process method by adding it
as a parameter:</p><div class="language-java snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><div class=highlight><pre class=chroma><code class=language-java data-lang=java><span class=o>.</span><span class=na>of</span><span class=o>(</span><span class=k>new</span> <span class=n>DoFn</span><span class=o><</span><span class=n>String</span><span class=o [...]