You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by gi...@apache.org on 2020/07/14 18:02:04 UTC
[beam] branch asf-site updated: Publishing website 2020/07/14
18:01:52 at commit c946886
This is an automated email from the ASF dual-hosted git repository.
git-site-role pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 56eb85a Publishing website 2020/07/14 18:01:52 at commit c946886
56eb85a is described below
commit 56eb85a6b126b197d86bfc699120289e71dfb90a
Author: jenkins <us...@infra.apache.org>
AuthorDate: Tue Jul 14 18:01:52 2020 +0000
Publishing website 2020/07/14 18:01:52 at commit c946886
---
.../pipelines/test-your-pipeline/index.html | 45 +---------------------
website/generated-content/sitemap.xml | 2 +-
2 files changed, 3 insertions(+), 44 deletions(-)
diff --git a/website/generated-content/documentation/pipelines/test-your-pipeline/index.html b/website/generated-content/documentation/pipelines/test-your-pipeline/index.html
index 03016c9..68fa566 100644
--- a/website/generated-content/documentation/pipelines/test-your-pipeline/index.html
+++ b/website/generated-content/documentation/pipelines/test-your-pipeline/index.html
@@ -1,47 +1,7 @@
<!doctype html><html lang=en class=no-js><head><meta charset=utf-8><meta http-equiv=x-ua-compatible content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1"><title>Test Your Pipeline</title><meta name=description content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Spe [...]
<span class=sr-only>Toggle navigation</span>
<span class=icon-bar></span><span class=icon-bar></span><span class=icon-bar></span></button>
-<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
- <span class=n>MyDoFn</span> <span class=n>myDoFn</span> <span class=o>=</span> <span class=o>...;</span>
-
- <span class=n>DoFnTester</span><span class=o><</span><span class=n>String</span><span class=o>,</span> <span class=n>Integer</span><span class=o>></span> <span class=n>fnTester</span> <span class=o>=</span> <span class=n>DoFnTester</span><span class=o>.</span><span class=na>of</span><span class=o>(</span><span class=n>myDoFn</span><span class=o>);</span></code></pre></div></div><h3 id=creating-test-inputs>Creating Test Inputs</h3><p>You’ll need to create one or more test in [...]
-<span class=n>MyDoFn</span> <span class=n>myDoFn</span> <span class=o>=</span> <span class=o>...;</span>
-<span class=n>DoFnTester</span><span class=o><</span><span class=n>String</span><span class=o>,</span> <span class=n>Integer</span><span class=o>></span> <span class=n>fnTester</span> <span class=o>=</span> <span class=n>DoFnTester</span><span class=o>.</span><span class=na>of</span><span class=o>(</span><span class=n>myDoFn</span><span class=o>);</span>
-
-<span class=n>String</span> <span class=n>testInput</span> <span class=o>=</span> <span class=s>"test1"</span><span class=o>;</span></code></pre></div></div><h4 id=side-inputs>Side Inputs</h4><p>If your <code>DoFn</code> accepts side inputs, you can create those side inputs by using the method <code>DoFnTester.setSideInputs</code>.</p><div class=language-java><div class=highlight><pre class=chroma><code class=language-java data-lang=java><span class=kd>static</span> <span class=k [...]
-<span class=n>MyDoFn</span> <span class=n>myDoFn</span> <span class=o>=</span> <span class=o>...;</span>
-<span class=n>DoFnTester</span><span class=o><</span><span class=n>String</span><span class=o>,</span> <span class=n>Integer</span><span class=o>></span> <span class=n>fnTester</span> <span class=o>=</span> <span class=n>DoFnTester</span><span class=o>.</span><span class=na>of</span><span class=o>(</span><span class=n>myDoFn</span><span class=o>);</span>
-
-<span class=n>PCollectionView</span><span class=o><</span><span class=n>List</span><span class=o><</span><span class=n>Integer</span><span class=o>>></span> <span class=n>sideInput</span> <span class=o>=</span> <span class=o>...;</span>
-<span class=n>Iterable</span><span class=o><</span><span class=n>Integer</span><span class=o>></span> <span class=n>value</span> <span class=o>=</span> <span class=o>...;</span>
-<span class=n>fnTester</span><span class=o>.</span><span class=na>setSideInputInGlobalWindow</span><span class=o>(</span><span class=n>sideInput</span><span class=o>,</span> <span class=n>value</span><span class=o>);</span></code></pre></div></div><p>See the <code>ParDo</code> documentation on <a href=/documentation/programming-guide/#side-inputs>side inputs</a> for more information.</p><h4 id=additional-outputs>Additional Outputs</h4><p>If your <code>DoFn</code> produces multiple output [...]
-appropriate <code>TupleTag</code> objects that you’ll use to access each output. A <code>DoFn</code>
-with multiple outputs produces a <code>PCollectionTuple</code> for each output; you’ll need
-to provide a <code>TupleTagList</code> that corresponds to each output in that tuple.</p><p>Suppose your <code>DoFn</code> produces outputs of type <code>String</code> and <code>Integer</code>. You create
-<code>TupleTag</code> objects for each, and bundle them into a <code>TupleTagList</code>, then set it
-for the <code>DoFnTester</code> as follows:</p><div class=language-java><div class=highlight><pre class=chroma><code class=language-java data-lang=java><span class=kd>static</span> <span class=kd>class</span> <span class=nc>MyDoFn</span> <span class=kd>extends</span> <span class=n>DoFn</span><span class=o><</span><span class=n>String</span><span class=o>,</span> <span class=n>Integer</span><span class=o>></span> <span class=o>{</span> <span class=o>...</span> <span class=o>}</span>
-<span class=n>MyDoFn</span> <span class=n>myDoFn</span> <span class=o>=</span> <span class=o>...;</span>
-<span class=n>DoFnTester</span><span class=o><</span><span class=n>String</span><span class=o>,</span> <span class=n>Integer</span><span class=o>></span> <span class=n>fnTester</span> <span class=o>=</span> <span class=n>DoFnTester</span><span class=o>.</span><span class=na>of</span><span class=o>(</span><span class=n>myDoFn</span><span class=o>);</span>
-
-<span class=n>TupleTag</span><span class=o><</span><span class=n>String</span><span class=o>></span> <span class=n>tag1</span> <span class=o>=</span> <span class=o>...;</span>
-<span class=n>TupleTag</span><span class=o><</span><span class=n>Integer</span><span class=o>></span> <span class=n>tag2</span> <span class=o>=</span> <span class=o>...;</span>
-<span class=n>TupleTagList</span> <span class=n>tags</span> <span class=o>=</span> <span class=n>TupleTagList</span><span class=o>.</span><span class=na>of</span><span class=o>(</span><span class=n>tag1</span><span class=o>).</span><span class=na>and</span><span class=o>(</span><span class=n>tag2</span><span class=o>);</span>
-
-<span class=n>fnTester</span><span class=o>.</span><span class=na>setOutputTags</span><span class=o>(</span><span class=n>tags</span><span class=o>);</span></code></pre></div></div><p>See the <code>ParDo</code> documentation on <a href=/documentation/programming-guide/#additional-outputs>additional outputs</a> for more information.</p><h3 id=processing-test-inputs-and-checking-results>Processing Test Inputs and Checking Results</h3><p>To process the inputs (and thus run the test on your [...]
-<span class=kd>static</span> <span class=kd>class</span> <span class=nc>MyDoFn</span> <span class=kd>extends</span> <span class=n>DoFn</span><span class=o><</span><span class=n>String</span><span class=o>,</span> <span class=n>Integer</span><span class=o>></span> <span class=o>{</span> <span class=o>...</span> <span class=o>}</span>
-<span class=n>MyDoFn</span> <span class=n>myDoFn</span> <span class=o>=</span> <span class=o>...;</span>
-<span class=n>DoFnTester</span><span class=o><</span><span class=n>String</span><span class=o>,</span> <span class=n>Integer</span><span class=o>></span> <span class=n>fnTester</span> <span class=o>=</span> <span class=n>DoFnTester</span><span class=o>.</span><span class=na>of</span><span class=o>(</span><span class=n>myDoFn</span><span class=o>);</span>
-
-<span class=n>String</span> <span class=n>testInput</span> <span class=o>=</span> <span class=s>"test1"</span><span class=o>;</span>
-<span class=n>List</span><span class=o><</span><span class=n>Integer</span><span class=o>></span> <span class=n>testOutputs</span> <span class=o>=</span> <span class=n>fnTester</span><span class=o>.</span><span class=na>processBundle</span><span class=o>(</span><span class=n>testInput</span><span class=o>);</span></code></pre></div></div><p>To check the results of <code>processBundle</code>, you use JUnit’s <code>Assert.assertThat</code> method to test if the <code>List</code [...]
-<span class=n>String</span> <span class=n>testInput</span> <span class=o>=</span> <span class=s>"test1"</span><span class=o>;</span>
-<span class=n>List</span><span class=o><</span><span class=n>Integer</span><span class=o>></span> <span class=n>testOutputs</span> <span class=o>=</span> <span class=n>fnTester</span><span class=o>.</span><span class=na>processBundle</span><span class=o>(</span><span class=n>testInput</span><span class=o>);</span>
-
-<span class=n>Assert</span><span class=o>.</span><span class=na>assertThat</span><span class=o>(</span><span class=n>testOutputs</span><span class=o>,</span> <span class=n>Matchers</span><span class=o>.</span><span class=na>hasItems</span><span class=o>(...));</span>
-
-<span class=c1>// Process a larger batch in a single step.
-</span><span class=c1></span><span class=n>Assert</span><span class=o>.</span><span class=na>assertThat</span><span class=o>(</span><span class=n>fnTester</span><span class=o>.</span><span class=na>processBundle</span><span class=o>(</span><span class=s>"input1"</span><span class=o>,</span> <span class=s>"input2"</span><span class=o>,</span> <span class=s>"input3"</span><span class=o>),</span> <span class=n>Matchers</span><span class=o>.</span><span class=na>hasIt [...]
-<span class=n>PCollection</span><span class=o><</span><span class=n>String</span><span class=o>></span> <span class=n>output</span> <span class=o>=</span> <span class=o>...;</span>
+<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
<span class=c1>// Check whether a PCollection contains some elements in any order.
</span><span class=c1></span><span class=n>PAssert</span><span class=o>.</span><span class=na>that</span><span class=o>(</span><span class=n>output</span><span class=o>)</span>
@@ -53,8 +13,7 @@ for the <code>DoFnTester</code> as follows:</p><div class=language-java><div cla
<span class=o><</span><span class=n>artifactId</span><span class=o>></span><span class=n>hamcrest</span><span class=o>-</span><span class=n>all</span><span class=o></</span><span class=n>artifactId</span><span class=o>></span>
<span class=o><</span><span class=n>version</span><span class=o>></span><span class=n>1</span><span class=o>.</span><span class=na>3</span><span class=o></</span><span class=n>version</span><span class=o>></span>
<span class=o><</span><span class=n>scope</span><span class=o>></span><span class=n>test</span><span class=o></</span><span class=n>scope</span><span class=o>></span>
-<span class=o></</span><span class=n>dependency</span><span class=o>></span></code></pre></div></div><p>For more information on how these classes work, see the <a href=https://beam.apache.org/releases/javadoc/2.22.0/index.html?org/apache/beam/sdk/testing/package-summary.html>org.apache.beam.sdk.testing</a> package documentation.</p><h3 id=an-example-test-for-a-composite-transform>An Example Test for a Composite Transform</h3><p>The following code shows a complete test for a composi [...]
-<span class=kd>public</span> <span class=kd>class</span> <span class=nc>CountTest</span> <span class=o>{</span>
+<span class=o></</span><span class=n>dependency</span><span class=o>></span></code></pre></div></div><p>For more information on how these classes work, see the <a href=https://beam.apache.org/releases/javadoc/2.22.0/index.html?org/apache/beam/sdk/testing/package-summary.html>org.apache.beam.sdk.testing</a> package documentation.</p><h3 id=an-example-test-for-a-composite-transform>An Example Test for a Composite Transform</h3><p>The following code shows a complete test for a composi [...]
<span class=c1>// Our static input data, which will make up the initial PCollection.
</span><span class=c1></span><span class=kd>static</span> <span class=kd>final</span> <span class=n>String</span><span class=o>[]</span> <span class=n>WORDS_ARRAY</span> <span class=o>=</span> <span class=k>new</span> <span class=n>String</span><span class=o>[]</span> <span class=o>{</span>
diff --git a/website/generated-content/sitemap.xml b/website/generated-content/sitemap.xml
index d197b45..47f25e5 100644
--- a/website/generated-content/sitemap.xml
+++ b/website/generated-content/sitemap.xml
@@ -1 +1 @@
-<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml"><url><loc>/blog/beam-2.22.0/</loc><lastmod>2020-06-08T14:13:37-07:00</lastmod></url><url><loc>/categories/blog/</loc><lastmod>2020-06-08T14:13:37-07:00</lastmod></url><url><loc>/blog/</loc><lastmod>2020-06-08T14:13:37-07:00</lastmod></url><url><loc>/categories/</loc><lastmod>2020-06-08T14:13:37-07:00</lastmod></url><url><loc>/blog/b [...]
\ No newline at end of file
+<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml"><url><loc>/blog/beam-2.22.0/</loc><lastmod>2020-06-08T14:13:37-07:00</lastmod></url><url><loc>/categories/blog/</loc><lastmod>2020-06-08T14:13:37-07:00</lastmod></url><url><loc>/blog/</loc><lastmod>2020-06-08T14:13:37-07:00</lastmod></url><url><loc>/categories/</loc><lastmod>2020-06-08T14:13:37-07:00</lastmod></url><url><loc>/blog/b [...]
\ No newline at end of file