You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Kenneth Knowles (Jira)" <ji...@apache.org> on 2021/11/08 19:59:00 UTC
[jira] [Updated] (BEAM-13161) BatchLoads setting wrong sideinput
[ https://issues.apache.org/jira/browse/BEAM-13161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kenneth Knowles updated BEAM-13161:
-----------------------------------
Description:
BigQueryIO failing to load data for multi partitions. Bug introduced in [https://github.com/apache/beam/commit/69822e417fe4b582a73c45c5780f4ff69841d5db#diff-8ec245029a6375b2ab19e1aff5e4ed4ee33a4a2e46a9caffb3452cbc8af8fee3. |https://github.com/apache/beam/commit/69822e417fe4b582a73c45c5780f4ff69841d5db#diff-8ec245029a6375b2ab19e1aff5e4ed4ee33a4a2e46a9caffb3452cbc8af8fee3,]missing setting correct sideinput (withSideInputs(copyJobIdPrefixView)).
Getting this error in DF job
{code}
Error message from worker: java.lang.IllegalArgumentException: calling sideInput() with unknown view org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.sideInput(SimpleDoFnRunner.java:265) org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.access$1200(SimpleDoFnRunner.java:84) org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.sideInput(SimpleDoFnRunner.java:395) org.apache.beam.sdk.io.gcp.bigquery.WriteRename.startWriteRename(WriteRename.java:207) org.apache.beam.sdk.io.gcp.bigquery.WriteRename.processElement(WriteRename.java:139)
{code}
was:
BigQueryIO failing to load data for multi partitions. Bug introduced in [https://github.com/apache/beam/commit/69822e417fe4b582a73c45c5780f4ff69841d5db#diff-8ec245029a6375b2ab19e1aff5e4ed4ee33a4a2e46a9caffb3452cbc8af8fee3. |https://github.com/apache/beam/commit/69822e417fe4b582a73c45c5780f4ff69841d5db#diff-8ec245029a6375b2ab19e1aff5e4ed4ee33a4a2e46a9caffb3452cbc8af8fee3,]missing setting correct sideinput (withSideInputs(copyJobIdPrefixView)).
Getting this error in DF job
Error message from worker: java.lang.IllegalArgumentException: calling sideInput() with unknown view org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.sideInput(SimpleDoFnRunner.java:265) org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.access$1200(SimpleDoFnRunner.java:84) org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.sideInput(SimpleDoFnRunner.java:395) org.apache.beam.sdk.io.gcp.bigquery.WriteRename.startWriteRename(WriteRename.java:207) org.apache.beam.sdk.io.gcp.bigquery.WriteRename.processElement(WriteRename.java:139)
> BatchLoads setting wrong sideinput
> -----------------------------------
>
> Key: BEAM-13161
> URL: https://issues.apache.org/jira/browse/BEAM-13161
> Project: Beam
> Issue Type: Bug
> Components: beam-model
> Reporter: Hector Lagos
> Priority: P2
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> BigQueryIO failing to load data for multi partitions. Bug introduced in [https://github.com/apache/beam/commit/69822e417fe4b582a73c45c5780f4ff69841d5db#diff-8ec245029a6375b2ab19e1aff5e4ed4ee33a4a2e46a9caffb3452cbc8af8fee3. |https://github.com/apache/beam/commit/69822e417fe4b582a73c45c5780f4ff69841d5db#diff-8ec245029a6375b2ab19e1aff5e4ed4ee33a4a2e46a9caffb3452cbc8af8fee3,]missing setting correct sideinput (withSideInputs(copyJobIdPrefixView)).
>
> Getting this error in DF job
> {code}
> Error message from worker: java.lang.IllegalArgumentException: calling sideInput() with unknown view org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.sideInput(SimpleDoFnRunner.java:265) org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.access$1200(SimpleDoFnRunner.java:84) org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.sideInput(SimpleDoFnRunner.java:395) org.apache.beam.sdk.io.gcp.bigquery.WriteRename.startWriteRename(WriteRename.java:207) org.apache.beam.sdk.io.gcp.bigquery.WriteRename.processElement(WriteRename.java:139)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)