[jira] [Commented] (BEAM-1112) Python E2E Integration Test Framework - Batch Only

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-1112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15753301#comment-15753301 ] 

ASF GitHub Bot commented on BEAM-1112:
--------------------------------------

GitHub user markflyhigh opened a pull request:

    https://github.com/apache/incubator-beam/pull/1639

    [BEAM-1112] Python E2E Test Framework And Wordcount E2E Test

    Be sure to do all of the following to help us incorporate your contribution
    quickly and easily:
    
     - [x] Make sure the PR title is formatted like:
       `[BEAM-<Jira issue #>] Description of pull request`
     - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
           Travis-CI on your fork and ensure the whole test matrix passes).
     - [ ] Replace `<Jira issue #>` in the title with the actual Jira issue
           number, if there is one.
     - [ ] If this contribution is large, please file an Apache
           [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt).
    
    ---
    
     - E2e test framework that supports TestRunner start and verify pipeline job.
       - add `TestOptions` which defined `on_success_matcher` that is used to verify state/output of pipeline job.
       - validate `on_success_matcher` before pipeline execution to make sure it's unpicklable to a subclass of BaseMatcher.
       - create a `TestDataflowRunner` which provide functionalities of `DataflowRunner` plus result verification.
       - provide a test verifier `PipelineStateMatcher` that can verify pipeline job finished in DONE or not.
     - Add wordcount_it (it = integration test) that build e2e test based on existing wordcount pipeline.
       - include wordcount_it to nose collector, so that wordcount_it can be collected and run by nose.
       - skip ITs when running unit tests from tox in precommit and postcommit.
    
    Current changes will not change behavior of existing pre/postcommit.
    Test is done by running `tox -e py27 -c sdks/python/tox.ini` for unit test and running wordcount_it with `TestDataflowRunner` on service ([link](https://pantheon.corp.google.com/dataflow/job/2016-12-15_17_36_16-3857167705491723621?project=google.com:clouddfe)).
    
    TODO:
     - Output data verifier that verify pipeline output that stores in filesystem.
     - Add wordcount_it to precommit and replace existing wordcount execution command in postcommit with a better structured nose command.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/markflyhigh/incubator-beam e2e-testrunner

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-beam/pull/1639.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1639
    
----
commit e1e1fa3a60e1fe234829432d144d6689e240b6f0
Author: Mark Liu <ma...@google.com>
Date:   2016-12-16T01:41:20Z

    [BEAM-1112] Python E2E Test Framework And Wordcount E2E Test

commit 0e7007879ee082e3afe5db36107f51c03274f3f5
Author: Mark Liu <ma...@google.com>
Date:   2016-12-16T02:55:53Z

    fixup! Fix Code Style

----


> Python E2E Integration Test Framework - Batch Only
> --------------------------------------------------
>
>                 Key: BEAM-1112
>                 URL: https://issues.apache.org/jira/browse/BEAM-1112
>             Project: Beam
>          Issue Type: Task
>          Components: sdk-py, testing
>            Reporter: Mark Liu
>            Assignee: Mark Liu
>
> Parity with Java. 
> Build e2e integration test framework that can configure and run batch pipeline with specified test runner, wait for pipeline execution and verify results with given verifiers in the end.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)