You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Robert Burke (Jira)" <ji...@apache.org> on 2021/06/01 17:40:00 UTC
[jira] [Created] (BEAM-12438) Sharing input to both a GBK and a
CoGBK causes LP error w/schemas
Robert Burke created BEAM-12438:
-----------------------------------
Summary: Sharing input to both a GBK and a CoGBK causes LP error w/schemas
Key: BEAM-12438
URL: https://issues.apache.org/jira/browse/BEAM-12438
Project: Beam
Issue Type: Improvement
Components: sdk-go
Reporter: Robert Burke
Assignee: Robert Burke
Constructing a pipeline with a PCol<KV> (call it A) used as an input to a GBK, using that output to generate an PCol<KV> (call it B) and using both A and B as inputs to a CoGBK causes the DoFn after the CoGBK to fail with an encoding error.
Essentially the Length Prefix coder applied to A isn't applied to the output of the CoGBK for the expand, causing the pipeline to fail. Only happens with Row encoded structs.
This is likely due to CoGBK being a synthetic construct on top of a GBK with a extra Inject and Expand operations, with the Inject missing the LP coding that the earlier GBK received.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)