You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Robert Burke (Jira)" <ji...@apache.org> on 2021/06/01 17:40:00 UTC

[jira] [Created] (BEAM-12438) Sharing input to both a GBK and a CoGBK causes LP error w/schemas

Robert Burke created BEAM-12438:
-----------------------------------

             Summary: Sharing input to both a GBK and a CoGBK causes LP error w/schemas
                 Key: BEAM-12438
                 URL: https://issues.apache.org/jira/browse/BEAM-12438
             Project: Beam
          Issue Type: Improvement
          Components: sdk-go
            Reporter: Robert Burke
            Assignee: Robert Burke


Constructing a pipeline with a PCol<KV> (call it A) used as an input to a GBK, using that output to generate an PCol<KV> (call it B) and using both A and B as inputs to a CoGBK causes the DoFn after the CoGBK to fail with an encoding error.

Essentially the Length Prefix coder applied to A isn't applied to the output of the CoGBK for the expand, causing the pipeline to fail. Only happens with Row encoded structs.

This is likely due to CoGBK being a synthetic construct on top of a GBK with a extra Inject and Expand operations, with the Inject missing the LP coding that the earlier GBK received.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)