You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by me...@apache.org on 2018/08/01 21:09:20 UTC
[beam-site] branch asf-site updated (6d7fd1a -> bcb7061)
This is an automated email from the ASF dual-hosted git repository.
mergebot-role pushed a change to branch asf-site
in repository https://gitbox.apache.org/repos/asf/beam-site.git.
from 6d7fd1a Prepare repository for deployment.
add 92a9016 Remove testing matrix and simplify intro.
add fc61efe This closes #516
new bcb7061 Prepare repository for deployment.
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
content/contribute/testing/index.html | 273 +---------------------------------
src/contribute/testing.md | 233 +----------------------------
2 files changed, 11 insertions(+), 495 deletions(-)
[beam-site] 01/01: Prepare repository for deployment.
Posted by me...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
mergebot-role pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/beam-site.git
commit bcb7061eb21de36e7ea353d4337fef6537e6db48
Author: Mergebot <me...@apache.org>
AuthorDate: Wed Aug 1 21:09:18 2018 +0000
Prepare repository for deployment.
---
content/contribute/testing/index.html | 273 +---------------------------------
1 file changed, 5 insertions(+), 268 deletions(-)
diff --git a/content/contribute/testing/index.html b/content/contribute/testing/index.html
index 85a50d8..333a387 100644
--- a/content/contribute/testing/index.html
+++ b/content/contribute/testing/index.html
@@ -28,7 +28,7 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Beam Testing Guide</title>
+ <title>Beam Testing</title>
<meta name="description" content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Specific Languages (DSLs). Dataflow pipelines simplify the mechanics of large-scale batch and streaming data processing and can run on a number of runtimes like Apache Flink, Apache Spark, and Google Cloud Dataflow [...]
">
<link href="https://fonts.googleapis.com/css?family=Roboto:100,300,400" rel="stylesheet">
@@ -195,13 +195,6 @@
<ul class="nav">
- <li><a href="#overview">Overview</a></li>
- <li><a href="#testing-matrix">Testing Matrix</a>
- <ul>
- <li><a href="#java-sdk">Java SDK</a></li>
- <li><a href="#python-sdk">Python SDK</a></li>
- </ul>
- </li>
<li><a href="#testing-scenarios">Testing Scenarios</a>
<ul>
<li><a href="#precommit">Precommit</a></li>
@@ -251,267 +244,11 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
-<h1 id="beam-testing-documentation">Beam Testing Documentation</h1>
-
-<ul id="markdown-toc">
- <li><a href="#overview" id="markdown-toc-overview">Overview</a></li>
- <li><a href="#testing-matrix" id="markdown-toc-testing-matrix">Testing Matrix</a> <ul>
- <li><a href="#java-sdk" id="markdown-toc-java-sdk">Java SDK</a></li>
- <li><a href="#python-sdk" id="markdown-toc-python-sdk">Python SDK</a></li>
- </ul>
- </li>
- <li><a href="#testing-scenarios" id="markdown-toc-testing-scenarios">Testing Scenarios</a> <ul>
- <li><a href="#precommit" id="markdown-toc-precommit">Precommit</a></li>
- <li><a href="#postcommit" id="markdown-toc-postcommit">Postcommit</a></li>
- </ul>
- </li>
- <li><a href="#testing-types" id="markdown-toc-testing-types">Testing Types</a> <ul>
- <li><a href="#unit" id="markdown-toc-unit">Unit</a> <ul>
- <li><a href="#how-to-run-python-unit-tests" id="markdown-toc-how-to-run-python-unit-tests">How to run Python unit tests</a></li>
- <li><a href="#how-to-run-java-needsrunner-tests" id="markdown-toc-how-to-run-java-needsrunner-tests">How to run Java NeedsRunner tests</a></li>
- </ul>
- </li>
- <li><a href="#validatesrunner" id="markdown-toc-validatesrunner">ValidatesRunner</a></li>
- <li><a href="#e2e" id="markdown-toc-e2e">E2E</a></li>
- </ul>
- </li>
- <li><a href="#testing-systems" id="markdown-toc-testing-systems">Testing Systems</a> <ul>
- <li><a href="#e2e-testing-framework" id="markdown-toc-e2e-testing-framework">E2E Testing Framework</a></li>
- <li><a href="#validatesrunner-tests" id="markdown-toc-validatesrunner-tests">ValidatesRunner Tests</a></li>
- <li><a href="#effective-use-of-the-testpipeline-junit-rule" id="markdown-toc-effective-use-of-the-testpipeline-junit-rule">Effective use of the TestPipeline JUnit rule</a></li>
- <li><a href="#api-surface-testing" id="markdown-toc-api-surface-testing">API Surface testing</a></li>
- </ul>
- </li>
- <li><a href="#best_practices" id="markdown-toc-best_practices">Best practices for writing tests</a> <ul>
- <li><a href="#aim-for-one-failure-path" id="markdown-toc-aim-for-one-failure-path">Aim for one failure path</a></li>
- <li><a href="#avoid-non-deterministic-code" id="markdown-toc-avoid-non-deterministic-code">Avoid non-deterministic code</a></li>
- <li><a href="#use-descriptive-test-names" id="markdown-toc-use-descriptive-test-names">Use descriptive test names</a></li>
- <li><a href="#use-a-pre-commit-test-if-possible" id="markdown-toc-use-a-pre-commit-test-if-possible">Use a pre-commit test if possible</a></li>
- </ul>
- </li>
-</ul>
+<h1 id="beam-testing">Beam Testing</h1>
-<h2 id="overview">Overview</h2>
-
-<p>Apache Beam is a rapidly-maturing software project with a strong
-commitment to testing. Consequently, it has many testing-related needs. It
-requires precommit tests to ensure code going into the repository meets a
-certain quality bar and it requires ongoing postcommit tests to make sure that
-more subtle changes which escape precommit are nonetheless caught. This document
-outlines how to write tests, which tests are appropriate where, and when tests
-are run, with some additional information about the testing systems at the
-bottom.</p>
-
-<p>If you’re writing tests, take a look at the testing matrix first, find what you
-want to test, then look into the “Scenarios” and “Types” sections below for more
-details on those testing types.</p>
-
-<h2 id="testing-matrix">Testing Matrix</h2>
-
-<h3 id="java-sdk">Java SDK</h3>
-
-<table>
- <tr>
- <td><strong>Component to Test</strong>
- </td>
- <td><strong>Test Scenario</strong>
- </td>
- <td><strong>Tool to Use</strong>
- </td>
- <td><strong>Link to Example</strong>
- </td>
- <td><strong>Type</strong>
- </td>
- <td><strong>Runs In</strong>
- </td>
- </tr>
- <td>BoundedSource
- </td>
- <td>Correctly Reads Input
- </td>
- <td><a href="https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/testing/SourceTestUtils.java#L128">SourceTestUtils.readFromSource</a>
- </td>
- <td><a href="https://github.com/apache/beam/blob/84a0dd1714028370befa80dea16f720edce05252/sdks/java/core/src/test/java/org/apache/beam/sdk/io/TextIOTest.java#L972">TextIOTest</a>
- </td>
- <td>Unit
- </td>
- <td>Precommit, Postcommit
- </td>
- </tr>
- <tr>
- <td>
- </td>
- <td>Correct Initial Splitting
- </td>
- <td><a href="https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/testing/SourceTestUtils.java#L201">SourceTestUtils.assertSourcesEqualReferenceSource</a>
- </td>
- <td><a href="https://github.com/apache/beam/blob/8b1e64a668489297e11926124c4eee6c8f69a3a7/sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableIOTest.java#L339">BigtableTest</a>
- </td>
- <td>Unit
- </td>
- <td>Precommit, Postcommit
- </td>
- </tr>
- <tr>
- <td>
- </td>
- <td>Correct Dynamic Splitting
- </td>
- <td><a href="https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/testing/SourceTestUtils.java#L541">SourceTestUtils. assertSplitAtFractionExhaustive</a>
- </td>
- <td><a href="https://github.com/apache/beam/blob/84a0dd1714028370befa80dea16f720edce05252/sdks/java/core/src/test/java/org/apache/beam/sdk/io/TextIOTest.java#L1021">TextIOTest</a>
- </td>
- <td>Unit
- </td>
- <td>Precommit, Postcommit
- </td>
- </tr>
- <tr>
- <td>Transform
- </td>
- <td>Correctness
- </td>
- <td>@NeedsRunner Test
- </td>
- <td><a href="https://github.com/apache/beam/blob/master/sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/ParDoTest.java#L1199">ParDoTest</a>
- </td>
- <td>@NeedsRunner
- </td>
- <td>
- </td>
- </tr>
- <tr>
- <td>Example Pipeline
- </td>
- <td>Verify Behavior on Each Runner
- </td>
- <td>E2E Test
- </td>
- <td><a href="https://github.com/apache/beam/blob/master/examples/java/src/test/java/org/apache/beam/examples/WordCountIT.java#L76">WordCountIT</a>
- </td>
- <td>E2E
- </td>
- <td>Postcommit (Except WordCountIT)
- </td>
- </tr>
- <tr>
- <td>Source/Sink with external resource
- </td>
- <td>External Resource Faked
- </td>
- <td>Unit / @NeedsRunner Test
- </td>
- <td><a href="https://github.com/apache/beam/blob/84a0dd1714028370befa80dea16f720edce05252/sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableIOTest.java#L646">FakeBigtableService in BigtableTest</a>
- </td>
- <td>Unit / @NeedsRunner
- </td>
- <td>Precommit / Postcommit
- </td>
- </tr>
- <tr>
- <td>
- </td>
- <td>Real Interactions With External Resource
- </td>
- <td>E2E Test
- </td>
- <td><a href="https://github.com/apache/beam/blob/84a0dd1714028370befa80dea16f720edce05252/sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableReadIT.java#L40">BigtableReadIT</a>
- </td>
- <td>E2E
- </td>
- <td>Postcommit
- </td>
- </tr>
- <tr>
- <td>Runner
- </td>
- <td>Correctness
- </td>
- <td>E2E Test, @ValidatesRunner
- </td>
- <td><a href="https://github.com/apache/beam/blob/master/examples/java/src/test/java/org/apache/beam/examples/WordCountIT.java#L78">WordCountIT</a>, <a href="https://github.com/apache/beam/blob/master/sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/ParDoTest.java">ParDoTest</a>
- </td>
- <td>E2E, @ValidatesRunner
- </td>
- <td>Postcommit
- </td>
- </tr>
- <tr>
- <td>Coders
- </td>
- <td>Encoding/decoding elements
- </td>
- <td><a href="https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/testing/CoderProperties.java">CoderProperties</a>
- </td>
- <td><a href="https://github.com/apache/beam/blob/master/sdks/java/core/src/test/java/org/apache/beam/sdk/coders/NullableCoderTest.java">NullableCoderTest</a>
- </td>
- <td>Unit
- </td>
- <td>Precommit / Postcommit
- </td>
- </tr>
- <tr>
- <td>
- </td>
- <td>Serialization/deserialization of Coder
- </td>
- <td><a href="https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/testing/CoderProperties.java">CoderProperties</a>
- </td>
- <td><a href="https://github.com/apache/beam/blob/master/sdks/java/core/src/test/java/org/apache/beam/sdk/coders/NullableCoderTest.java">NullableCoderTest</a>
- </td>
- <td>Unit
- </td>
- <td>Precommit / Postcommit
- </td>
- </tr>
- <tr>
- <td>
- </td>
- <td>Sizing of elements
- </td>
- <td><a href="https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/testing/CoderProperties.java">CoderProperties</a>
- </td>
- <td><a href="https://github.com/apache/beam/blob/master/sdks/java/core/src/test/java/org/apache/beam/sdk/coders/NullableCoderTest.java">NullableCoderTest</a>
- </td>
- <td>Unit
- </td>
- <td>Precommit / Postcommit
- </td>
- </tr>
- <tr>
- <td>
- </td>
- <td>Deterministic
- </td>
- <td><a href="https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/testing/CoderProperties.java">CoderProperties</a>
- </td>
- <td><a href="https://github.com/apache/beam/blob/master/sdks/java/core/src/test/java/org/apache/beam/sdk/coders/NullableCoderTest.java">NullableCoderTest</a>
- </td>
- <td>Unit
- </td>
- <td>Precommit / Postcommit
- </td>
- </tr>
- <tr>
- <td>
- </td>
- <td>Structural value equality
- </td>
- <td><a href="https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/testing/CoderProperties.java">CoderProperties</a>
- </td>
- <td><a href="https://github.com/apache/beam/blob/master/sdks/java/core/src/test/java/org/apache/beam/sdk/coders/NullableCoderTest.java">NullableCoderTest</a>
- </td>
- <td>Unit
- </td>
- <td>Precommit / Postcommit
- </td>
- </tr>
-</table>
-
-<h3 id="python-sdk">Python SDK</h3>
-
-<p>The Python SDK has postcommit tests by a Jenkins build; precommit testing and a
-full testing matrix will be coming soon.</p>
+<p>This document outlines how to write tests, which tests are appropriate where,
+and when tests are run, with some additional information about the testing
+systems at the bottom.</p>
<h2 id="testing-scenarios">Testing Scenarios</h2>