You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@beam.apache.org by "Kenneth Knowles (Jira)" <ji...@apache.org> on 2019/11/02 20:23:00 UTC

[jira] [Commented] (BEAM-8520) Expansion of _CombinePerKeyWithHotKeyFanout has no typehints

    [ https://issues.apache.org/jira/browse/BEAM-8520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16965490#comment-16965490 ] 

Kenneth Knowles commented on BEAM-8520:
---------------------------------------

CC [~udim]

> Expansion of _CombinePerKeyWithHotKeyFanout has no typehints
> ------------------------------------------------------------
>
>                 Key: BEAM-8520
>                 URL: https://issues.apache.org/jira/browse/BEAM-8520
>             Project: Beam
>          Issue Type: Bug
>          Components: sdk-py-core
>            Reporter: Yifan Mai
>            Priority: Major
>
> The expansion of [_CombinePerKeyWithHotKeyFanout|https://github.com/apache/beam/blob/bd09a038f4df896ef6d903a4944083fcd2e8f727/sdks/python/apache_beam/transforms/core.py#L1913], which is used by [CombinePerKey.with_hot_key_fanout()|https://github.com/apache/beam/blob/bd09a038f4df896ef6d903a4944083fcd2e8f727/sdks/python/apache_beam/transforms/core.py#L1734], uses DoFns (SplitHotCold) and CombineFns (PreCombineFn, PostCombineFn) and a lambda (SplitNonce) that have no typehints. If the user specified a typehints on the original CombineFn, the typehints are lost. This means that the fallback coder rather than the user specified coder.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)