[jira] [Commented] (BEAM-436) DirectRunner RunnableOnService tempLocation configuration insufficient

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15398157#comment-15398157 ] 

ASF GitHub Bot commented on BEAM-436:
-------------------------------------

GitHub user swegner opened a pull request:

    https://github.com/apache/incubator-beam/pull/753

    [BEAM-436] Re-enable RunnableOnService tests which need tempRoot

    Be sure to do all of the following to help us incorporate your contribution
    quickly and easily:
    
     - [ ] Make sure the PR title is formatted like:
       `[BEAM-<Jira issue #>] Description of pull request`
     - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
           Travis-CI on your fork and ensure the whole test matrix passes).
     - [ ] Replace `<Jira issue #>` in the title with the actual Jira issue
           number, if there is one.
     - [ ] If this contribution is large, please file an Apache
           [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt).
    
    ---
    
    @peihe's recent work to improve tempRoot handling means that these tests are now working properly. They can be re-enabled to validate runner display data for i/o transforms.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/swegner/incubator-beam io-runnableonservice

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-beam/pull/753.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #753
    
----
commit e741f32184838f598d351117e024995ff4f5d3fe
Author: Scott Wegner <sw...@google.com>
Date:   2016-07-28T20:35:55Z

    Re-enable RunnableOnService tests which need tempRoot

----


> DirectRunner RunnableOnService tempLocation configuration insufficient
> ----------------------------------------------------------------------
>
>                 Key: BEAM-436
>                 URL: https://issues.apache.org/jira/browse/BEAM-436
>             Project: Beam
>          Issue Type: Bug
>          Components: runner-direct
>            Reporter: Scott Wegner
>            Assignee: Davor Bonaci
>
> The way we specify temp storage locations for RunnableOnService tests is not sufficient, specifically for DirectRunner execution. Right now, RunnableOnService tests are run for DirectRunner and DataflowRunner, who set their temp locations differently:
> * DirectRunner doesn't specify a temp location directly, but test classes will use a JUnit @Rule TemporaryDirectory. Individual tests set it as necessary for tempLocation, and set a fake gs:// path for individual GCP IO tests.
> * DataflowRunner tests pass an actual GCS path as tempRoot, and TestDataflowRunner will initialize stagingLocation to this path.
> This setup makes it difficult to write RunnableOnService tests which pass for both runners. We should separate temp location setup out of individual test classes so that RunnableOnService tests "just work" on any runner.
> One solution would be to add logic inside TestPipeline#testingPipelineOptions:
> # If --tempRoot is specified, use it to set tempLocation and stagingLocation. Otherwise, use a JUnit TemporaryDirectory to set it
> # If tempLocation is a GCS path, use it to set stagingLocation. Otherwise, use a fake gcs path (i.e. gs://foo)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)