You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@beam.apache.org by "Paul Gerver (JIRA)" <ji...@apache.org> on 2018/01/24 15:42:00 UTC

[jira] [Created] (BEAM-3525) TestPipeline serializes PipelineOptions prematurely

Paul Gerver created BEAM-3525:
---------------------------------

             Summary: TestPipeline serializes PipelineOptions prematurely
                 Key: BEAM-3525
                 URL: https://issues.apache.org/jira/browse/BEAM-3525
             Project: Beam
          Issue Type: Bug
          Components: testing
    Affects Versions: 2.2.0
            Reporter: Paul Gerver
            Assignee: Paul Gerver


The TestPipeline in its run() method now serializes PipelineOptions and adds in TestValueProvider values before submitting the pipeline to run [1].


This premature serialization can cause options marked with JsonIgnore annotations to be dropped before a runner has had a chance to see those parameters. For example, take the FlinkRunner options, if flinkMaster is marked with JsonIgnore because that information is only needed by the runner submitting the job and is not needed during runtime, tests will be run locally incorrectly instead of against the Flink cluster.

[1] https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/testing/TestPipeline.java#L354



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)