You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by me...@apache.org on 2018/08/01 21:05:21 UTC
[beam-site] 01/02: Remove testing matrix and simplify intro.
This is an automated email from the ASF dual-hosted git repository.
mergebot-role pushed a commit to branch mergebot
in repository https://gitbox.apache.org/repos/asf/beam-site.git
commit 92a9016de82c61537a0ae43c506dd3ac7f65723c
Author: Udi Meiri <eh...@google.com>
AuthorDate: Mon Jul 30 16:02:30 2018 -0700
Remove testing matrix and simplify intro.
---
src/contribute/testing.md | 233 ++--------------------------------------------
1 file changed, 6 insertions(+), 227 deletions(-)
diff --git a/src/contribute/testing.md b/src/contribute/testing.md
index 5fdd9ac..ef0814b 100644
--- a/src/contribute/testing.md
+++ b/src/contribute/testing.md
@@ -1,6 +1,6 @@
---
layout: section
-title: 'Beam Testing Guide'
+title: 'Beam Testing'
section_menu: section-menu/contribute.html
permalink: /contribute/testing/
---
@@ -18,232 +18,11 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
-# Beam Testing Documentation
-
-* TOC
-{:toc}
-
-## Overview
-
-Apache Beam is a rapidly-maturing software project with a strong
-commitment to testing. Consequently, it has many testing-related needs. It
-requires precommit tests to ensure code going into the repository meets a
-certain quality bar and it requires ongoing postcommit tests to make sure that
-more subtle changes which escape precommit are nonetheless caught. This document
-outlines how to write tests, which tests are appropriate where, and when tests
-are run, with some additional information about the testing systems at the
-bottom.
-
-If you’re writing tests, take a look at the testing matrix first, find what you
-want to test, then look into the “Scenarios” and “Types” sections below for more
-details on those testing types.
-
-## Testing Matrix
-
-### Java SDK
-
-<table>
- <tr>
- <td><strong>Component to Test</strong>
- </td>
- <td><strong>Test Scenario</strong>
- </td>
- <td><strong>Tool to Use</strong>
- </td>
- <td><strong>Link to Example</strong>
- </td>
- <td><strong>Type</strong>
- </td>
- <td><strong>Runs In</strong>
- </td>
- </tr>
- <td>BoundedSource
- </td>
- <td>Correctly Reads Input
- </td>
- <td><a href="https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/testing/SourceTestUtils.java#L128">SourceTestUtils.readFromSource</a>
- </td>
- <td><a href="https://github.com/apache/beam/blob/84a0dd1714028370befa80dea16f720edce05252/sdks/java/core/src/test/java/org/apache/beam/sdk/io/TextIOTest.java#L972">TextIOTest</a>
- </td>
- <td>Unit
- </td>
- <td>Precommit, Postcommit
- </td>
- </tr>
- <tr>
- <td>
- </td>
- <td>Correct Initial Splitting
- </td>
- <td><a href="https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/testing/SourceTestUtils.java#L201">SourceTestUtils.assertSourcesEqualReferenceSource</a>
- </td>
- <td><a href="https://github.com/apache/beam/blob/8b1e64a668489297e11926124c4eee6c8f69a3a7/sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableIOTest.java#L339">BigtableTest</a>
- </td>
- <td>Unit
- </td>
- <td>Precommit, Postcommit
- </td>
- </tr>
- <tr>
- <td>
- </td>
- <td>Correct Dynamic Splitting
- </td>
- <td><a href="https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/testing/SourceTestUtils.java#L541">SourceTestUtils. assertSplitAtFractionExhaustive</a>
- </td>
- <td><a href="https://github.com/apache/beam/blob/84a0dd1714028370befa80dea16f720edce05252/sdks/java/core/src/test/java/org/apache/beam/sdk/io/TextIOTest.java#L1021">TextIOTest</a>
- </td>
- <td>Unit
- </td>
- <td>Precommit, Postcommit
- </td>
- </tr>
- <tr>
- <td>Transform
- </td>
- <td>Correctness
- </td>
- <td>@NeedsRunner Test
- </td>
- <td><a href="https://github.com/apache/beam/blob/master/sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/ParDoTest.java#L1199">ParDoTest</a>
- </td>
- <td>@NeedsRunner
- </td>
- <td>
- </td>
- </tr>
- <tr>
- <td>Example Pipeline
- </td>
- <td>Verify Behavior on Each Runner
- </td>
- <td>E2E Test
- </td>
- <td><a href="https://github.com/apache/beam/blob/master/examples/java/src/test/java/org/apache/beam/examples/WordCountIT.java#L76">WordCountIT</a>
- </td>
- <td>E2E
- </td>
- <td>Postcommit (Except WordCountIT)
- </td>
- </tr>
- <tr>
- <td>Source/Sink with external resource
- </td>
- <td>External Resource Faked
- </td>
- <td>Unit / @NeedsRunner Test
- </td>
- <td><a href="https://github.com/apache/beam/blob/84a0dd1714028370befa80dea16f720edce05252/sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableIOTest.java#L646">FakeBigtableService in BigtableTest</a>
- </td>
- <td>Unit / @NeedsRunner
- </td>
- <td>Precommit / Postcommit
- </td>
- </tr>
- <tr>
- <td>
- </td>
- <td>Real Interactions With External Resource
- </td>
- <td>E2E Test
- </td>
- <td><a href="https://github.com/apache/beam/blob/84a0dd1714028370befa80dea16f720edce05252/sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableReadIT.java#L40">BigtableReadIT</a>
- </td>
- <td>E2E
- </td>
- <td>Postcommit
- </td>
- </tr>
- <tr>
- <td>Runner
- </td>
- <td>Correctness
- </td>
- <td>E2E Test, @ValidatesRunner
- </td>
- <td><a href="https://github.com/apache/beam/blob/master/examples/java/src/test/java/org/apache/beam/examples/WordCountIT.java#L78">WordCountIT</a>, <a href="https://github.com/apache/beam/blob/master/sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/ParDoTest.java">ParDoTest</a>
- </td>
- <td>E2E, @ValidatesRunner
- </td>
- <td>Postcommit
- </td>
- </tr>
- <tr>
- <td>Coders
- </td>
- <td>Encoding/decoding elements
- </td>
- <td><a href="https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/testing/CoderProperties.java">CoderProperties</a>
- </td>
- <td><a href="https://github.com/apache/beam/blob/master/sdks/java/core/src/test/java/org/apache/beam/sdk/coders/NullableCoderTest.java">NullableCoderTest</a>
- </td>
- <td>Unit
- </td>
- <td>Precommit / Postcommit
- </td>
- </tr>
- <tr>
- <td>
- </td>
- <td>Serialization/deserialization of Coder
- </td>
- <td><a href="https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/testing/CoderProperties.java">CoderProperties</a>
- </td>
- <td><a href="https://github.com/apache/beam/blob/master/sdks/java/core/src/test/java/org/apache/beam/sdk/coders/NullableCoderTest.java">NullableCoderTest</a>
- </td>
- <td>Unit
- </td>
- <td>Precommit / Postcommit
- </td>
- </tr>
- <tr>
- <td>
- </td>
- <td>Sizing of elements
- </td>
- <td><a href="https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/testing/CoderProperties.java">CoderProperties</a>
- </td>
- <td><a href="https://github.com/apache/beam/blob/master/sdks/java/core/src/test/java/org/apache/beam/sdk/coders/NullableCoderTest.java">NullableCoderTest</a>
- </td>
- <td>Unit
- </td>
- <td>Precommit / Postcommit
- </td>
- </tr>
- <tr>
- <td>
- </td>
- <td>Deterministic
- </td>
- <td><a href="https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/testing/CoderProperties.java">CoderProperties</a>
- </td>
- <td><a href="https://github.com/apache/beam/blob/master/sdks/java/core/src/test/java/org/apache/beam/sdk/coders/NullableCoderTest.java">NullableCoderTest</a>
- </td>
- <td>Unit
- </td>
- <td>Precommit / Postcommit
- </td>
- </tr>
- <tr>
- <td>
- </td>
- <td>Structural value equality
- </td>
- <td><a href="https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/testing/CoderProperties.java">CoderProperties</a>
- </td>
- <td><a href="https://github.com/apache/beam/blob/master/sdks/java/core/src/test/java/org/apache/beam/sdk/coders/NullableCoderTest.java">NullableCoderTest</a>
- </td>
- <td>Unit
- </td>
- <td>Precommit / Postcommit
- </td>
- </tr>
-</table>
-
-### Python SDK
-
-The Python SDK has postcommit tests by a Jenkins build; precommit testing and a
-full testing matrix will be coming soon.
+# Beam Testing
+
+This document outlines how to write tests, which tests are appropriate where,
+and when tests are run, with some additional information about the testing
+systems at the bottom.
## Testing Scenarios