You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Etienne Chauchot (JIRA)" <ji...@apache.org> on 2019/02/14 09:47:00 UTC
[jira] [Updated] (BEAM-6663) Add the ability to transform to/from
json SerializablePipelineOptions
[ https://issues.apache.org/jira/browse/BEAM-6663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Etienne Chauchot updated BEAM-6663:
-----------------------------------
Description: Many runners, store both PipelineOptions and SerializablePipelineOptions in their context whereas SerializablePipelineOptions stores both Options and json serialized Options. Options can be obtain by SerializablePipelineOptions.get() but json form cannot be obtained. Similarly, we cannot create a SerializablePipelineOptions from json. It is needed in some cases (like spark structured streaming) where we cannot inject anything else than Strings into the framework. (was: Many runners, store both options and SerializableOptions in their context whereas SerializableOptions stores both Options and json serialized Options. Options can be obtain by SerializableOptions.get() and json cannot be obtained. I propose to use only SerializableOptions inside the runners (as they all have serialization issues) and simply override toString to get the json version.)
Summary: Add the ability to transform to/from json SerializablePipelineOptions (was: SerializablePipelineOptions should override toString())
> Add the ability to transform to/from json SerializablePipelineOptions
> ---------------------------------------------------------------------
>
> Key: BEAM-6663
> URL: https://issues.apache.org/jira/browse/BEAM-6663
> Project: Beam
> Issue Type: Improvement
> Components: runner-core
> Reporter: Etienne Chauchot
> Assignee: Etienne Chauchot
> Priority: Minor
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Many runners, store both PipelineOptions and SerializablePipelineOptions in their context whereas SerializablePipelineOptions stores both Options and json serialized Options. Options can be obtain by SerializablePipelineOptions.get() but json form cannot be obtained. Similarly, we cannot create a SerializablePipelineOptions from json. It is needed in some cases (like spark structured streaming) where we cannot inject anything else than Strings into the framework.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)