You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by "Thomas Groh (JIRA)" <ji...@apache.org> on 2017/06/15 22:18:00 UTC
[jira] [Created] (BEAM-2453) The DirectRunner should exercise all
parts of a CombineFn
Thomas Groh created BEAM-2453:
---------------------------------
Summary: The DirectRunner should exercise all parts of a CombineFn
Key: BEAM-2453
URL: https://issues.apache.org/jira/browse/BEAM-2453
Project: Beam
Issue Type: Bug
Components: runner-direct
Reporter: Thomas Groh
Assignee: Thomas Groh
Specifically it should:
Create some number of accumulators; add elements to these accumulators, merge the created accumulators, and extract the output.
This can be performed by replacing the {{Combine.perKey}} composite transform with a multi-step combine {{CombineBundles -> GroupByKey -> MergeAccumulators}}
Where {{CombineBundles}} is a {{ParDo}} which takes input {{KV<K, InputT>}} and produces {{KV<K, AccumT>}}, outputting in {{FinishBundle}} (this can only be performed if the Combine takes no side inputs or does not have merging windows). {{MergeAccumulators}} takes in {{KV<K, Iterable<AccumT>>}} and produces {{KV<K, OutputT>}} by merging all of the accumulators and extracting the output.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)