You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@beam.apache.org by "Yifan Mai (Jira)" <ji...@apache.org> on 2021/02/18 22:49:00 UTC

[jira] [Commented] (BEAM-10641) Add Combiner Packing to Dataflow

    [ https://issues.apache.org/jira/browse/BEAM-10641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17286756#comment-17286756 ] 

Yifan Mai commented on BEAM-10641:
----------------------------------

Unfortunately combiner packing for Dataflow was rolled back in https://github.com/apache/beam/pull/13884 because it was producing a CombinePerKey that was a primitive and had no subtransforms, and Dataflow is unable to execute such a CombinePerKey.

> Add Combiner Packing to Dataflow
> --------------------------------
>
>                 Key: BEAM-10641
>                 URL: https://issues.apache.org/jira/browse/BEAM-10641
>             Project: Beam
>          Issue Type: Improvement
>          Components: sdk-py-core
>            Reporter: Yifan Mai
>            Priority: P3
>          Time Spent: 6.5h
>  Remaining Estimate: 0h
>
> BEAM-10409 adds a combiner packing graph optimizer for PortableRunner and FnApiRunner. We should also add this to the Dataflow runner.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)