You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Chamikara Jayalath (JIRA)" <ji...@apache.org> on 2019/03/23 15:14:00 UTC
[jira] [Commented] (BEAM-6894) ExternalTransform.expand() does not
create the proper AppliedPTransform sub-graph
[ https://issues.apache.org/jira/browse/BEAM-6894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16799670#comment-16799670 ]
Chamikara Jayalath commented on BEAM-6894:
------------------------------------------
[~robertwb], do you have any ideas on how to properly add this missing piece ? I was thinking about building the top level ExternalTransform object using "ptransform.PTransform.from_runner_api() " and adding the sub-graph starting from to parent 'AppliedPTransform' by applying into input pvalue. But I ran into an issue here since 'from_runner_api()' does not seem to be building the PTransform hierarchy for 'ExternalTransform' correctly.
> ExternalTransform.expand() does not create the proper AppliedPTransform sub-graph
> ---------------------------------------------------------------------------------
>
> Key: BEAM-6894
> URL: https://issues.apache.org/jira/browse/BEAM-6894
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
> Reporter: Chamikara Jayalath
> Priority: Major
>
> 'ExternalTransform.expand()' can be used to expand a remote transform and build the correct runner-api subgraph for that transform. However currently we do not modify the AppliedPTransform sub-graph correctly during this process. Relevant code location here.
> [https://github.com/apache/beam/blob/master/sdks/python/apache_beam/transforms/external.py#L135]
>
> Without this, DataflowRunner that relies in this object graph (not just the runner API proto) to build the job submission request to Dataflow service cannot construct this request properly.
>
> cc: [~robertwb]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)