[jira] [Closed] (BEAM-3525) TestPipeline serializes PipelineOptions prematurely

["Paul Gerver (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/BEAM-3525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Paul Gerver closed BEAM-3525.
-----------------------------
       Resolution: Workaround
    Fix Version/s: Not applicable

PipelineOptions need to be serialized for non-Java SDKs before they get to the runner. As such, options that need to be present in the runner should not be marked with @JsonIgnore. 

Solution: Remove JsonIgnore annotation from runner configuration options.

> TestPipeline serializes PipelineOptions prematurely
> ---------------------------------------------------
>
>                 Key: BEAM-3525
>                 URL: https://issues.apache.org/jira/browse/BEAM-3525
>             Project: Beam
>          Issue Type: Bug
>          Components: testing
>    Affects Versions: 2.2.0
>            Reporter: Paul Gerver
>            Assignee: Paul Gerver
>            Priority: Minor
>             Fix For: Not applicable
>
>
> The TestPipeline in its run() method now serializes PipelineOptions and adds in TestValueProvider values before submitting the pipeline to run [1].
> This premature serialization can cause options marked with JsonIgnore annotations to be dropped before a runner has had a chance to see those parameters. For example, take the FlinkRunner options, if flinkMaster is marked with JsonIgnore because that information is only needed by the runner submitting the job and is not needed during runtime, tests will be run locally incorrectly instead of against the Flink cluster.
> [1] https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/testing/TestPipeline.java#L354



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)