You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Jeff Klukas (Jira)" <ji...@apache.org> on 2019/10/14 13:34:00 UTC

[jira] [Created] (BEAM-8393) Java BigQueryIO clustering support breaks on multiple partitions

Jeff Klukas created BEAM-8393:
---------------------------------

             Summary: Java BigQueryIO clustering support breaks on multiple partitions
                 Key: BEAM-8393
                 URL: https://issues.apache.org/jira/browse/BEAM-8393
             Project: Beam
          Issue Type: Bug
          Components: io-java-gcp
    Affects Versions: 2.15.0, 2.16.0
            Reporter: Jeff Klukas
            Assignee: Jeff Klukas


Support for writing to clustered tables in BigQuery was added in 2.15, which involved adding a new optional clustering field to TableDestination. Clustering support is working for most cases, but fails with errors about incompatible partitioning specifications for any data that is handled by the MultiplePartitions branch of BigQueryIO logic.

There is a case in that code path where we provide a modified TableDestination and neglect to copy the clustering definition, so the final load job does not include any clustering columns.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)