[jira] [Commented] (BEAM-6294) Use Flink's redistribute for reshuffle.

["Robert Bradshaw (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-6294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16738323#comment-16738323 ] 

Robert Bradshaw commented on BEAM-6294:
---------------------------------------

Basically, I'm letting the existing reshuffle URN transform do its thing in more places (specifically, in Python). Is this what is breaking your pipelines? 

As for test coverage, isn't this covered by the existing reshuffle tests? (I also unified the streaming vs. batch translation code a bit to avoid duplicating even more code). 

> Use Flink's redistribute for reshuffle.
> ---------------------------------------
>
>                 Key: BEAM-6294
>                 URL: https://issues.apache.org/jira/browse/BEAM-6294
>             Project: Beam
>          Issue Type: New Feature
>          Components: runner-flink, sdk-py-core
>            Reporter: Robert Bradshaw
>            Assignee: Robert Bradshaw
>            Priority: Major
>             Fix For: 2.10.0
>
>          Time Spent: 1h
>  Remaining Estimate: 0h
>
> Python needs to publish the URN over the FnAPI which is pretty easy, but Flink also needs to ensure that the composite structure does not get fused. Unlike with GBK, we can't assume all runners implement this as a primitive. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)