You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@beam.apache.org by "Luke Cwik (JIRA)" <ji...@apache.org> on 2018/05/09 16:49:00 UTC

[jira] [Commented] (BEAM-4236) Side inputs do not work on Dataflow

    [ https://issues.apache.org/jira/browse/BEAM-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16469072#comment-16469072 ] 

Luke Cwik commented on BEAM-4236:
---------------------------------

Several issues that were incomplete within the Python SDK:
* Lack of support for mapping windows
* relying on reading the side input specification that was being serialized within the DoFn and not read from the pipeline description. Removing this required a number of changes to deprecate usage of tags_and_types throughout the Python SDK.
* Eagerly pre-fetching the side input when the process bundle had zero elements which lead to a state request for a non-existent side input.

These issues have been addressed in https://github.com/apache/beam/pull/5302

> Side inputs do not work on Dataflow
> -----------------------------------
>
>                 Key: BEAM-4236
>                 URL: https://issues.apache.org/jira/browse/BEAM-4236
>             Project: Beam
>          Issue Type: Sub-task
>          Components: sdk-py-harness
>    Affects Versions: Not applicable
>            Reporter: Charles Chen
>            Assignee: Luke Cwik
>            Priority: Blocker
>             Fix For: 2.5.0
>
>
> Dataflow side inputs do not work in streaming / batch FnApi mode at head.  The pipeline stalls indefinitely.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)