You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Daniel Oliveira (Jira)" <ji...@apache.org> on 2020/04/13 19:20:00 UTC
[jira] [Updated] (BEAM-9745) [beam_PostCommit_Java_PortabilityApi]
Various GCP IO tests failing, unable to deserialize Custom DoFns and Custom
Coders.
[ https://issues.apache.org/jira/browse/BEAM-9745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Oliveira updated BEAM-9745:
----------------------------------
Description:
_Use this form to file an issue for test failure:_
* [Jenkins Job|https://builds.apache.org/job/beam_PostCommit_Java_PortabilityApi/4657/]
* [Gradle Build Scan|https://scans.gradle.com/s/c3izncsa4u24k/tests/by-project]
Initial investigation:
The bug appears to be popping up on BigQuery tests mostly, but also a BigTable and a Datastore test.
Here's an example stacktrace of the two errors, showing _only_ the error messages themselves. Source: [https://scans.gradle.com/s/c3izncsa4u24k/tests/efn4wciuamvqq-ccxt3jvofvqbe]
{noformat}
java.util.concurrent.ExecutionException: java.lang.RuntimeException: Error received from SDK harness for instruction -191: java.lang.IllegalArgumentException: unable to deserialize Custom DoFn With Execution Info
...
Caused by: java.lang.ClassNotFoundException: org.apache.beam.sdk.io.gcp.bigquery.BatchLoads$3
...
Caused by: java.lang.RuntimeException: Error received from SDK harness for instruction -191: java.lang.IllegalArgumentException: unable to deserialize Custom DoFn With Execution Info
...
Caused by: java.lang.ClassNotFoundException: org.apache.beam.sdk.io.gcp.bigquery.BatchLoads$3
java.util.concurrent.ExecutionException: java.lang.RuntimeException: Error received from SDK harness for instruction -206: org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException: org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException: java.lang.IllegalArgumentException: unable to deserialize Custom Coder Bytes
...
Caused by: org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException: java.lang.IllegalArgumentException: unable to deserialize Custom Coder Bytes
...
Caused by: java.lang.IllegalArgumentException: unable to deserialize Custom Coder Bytes
...
Caused by: java.lang.ClassNotFoundException: org.apache.beam.sdk.io.gcp.bigquery.TableRowJsonCoder
...
Caused by: java.lang.RuntimeException: Error received from SDK harness for instruction -206: org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException: org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException: java.lang.IllegalArgumentException: unable to deserialize Custom Coder Bytes
...
Caused by: org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException: java.lang.IllegalArgumentException: unable to deserialize Custom Coder Bytes
...
Caused by: java.lang.IllegalArgumentException: unable to deserialize Custom Coder Bytes
...
Caused by: java.lang.ClassNotFoundException: org.apache.beam.sdk.io.gcp.bigquery.TableRowJsonCoder
...
{noformat}
Update: Looks like this has been failing as far back as [Apr 4|https://builds.apache.org/job/beam_PostCommit_Java_PortabilityApi/4566/] after a long period where the test was consistently timing out since [Mar 31|https://builds.apache.org/job/beam_PostCommit_Java_PortabilityApi/4546/]. So it's hard to narrow down what commit may have caused this. Plus, the test was failing due to a completely different BigQuery failure before anyway, so it seems like this test will need to be completely fixed from scratch, instead of tracking down a specific breaking change.
----
_After you've filled out the above details, please [assign the issue to an individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist]. Assignee should [treat test failures as high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test], helping to fix the issue or find a more appropriate owner. See [Apache Beam Post-Commit Policies|https://beam.apache.org/contribute/postcommits-policies]._
was:
_Use this form to file an issue for test failure:_
* [Jenkins Job|https://builds.apache.org/job/beam_PostCommit_Java_PortabilityApi/4657/]
* [Gradle Build Scan|https://scans.gradle.com/s/c3izncsa4u24k/tests/by-project]
Initial investigation:
The bug appears to be popping up on BigQuery tests mostly, but also a BigTable and a Datastore test.
Here's an example stacktrace of the two errors, showing _only_ the error messages themselves. Source: [https://scans.gradle.com/s/c3izncsa4u24k/tests/efn4wciuamvqq-ccxt3jvofvqbe]
{noformat}
java.util.concurrent.ExecutionException: java.lang.RuntimeException: Error received from SDK harness for instruction -191: java.lang.IllegalArgumentException: unable to deserialize Custom DoFn With Execution Info
...
Caused by: java.lang.ClassNotFoundException: org.apache.beam.sdk.io.gcp.bigquery.BatchLoads$3
...
Caused by: java.lang.RuntimeException: Error received from SDK harness for instruction -191: java.lang.IllegalArgumentException: unable to deserialize Custom DoFn With Execution Info
...
Caused by: java.lang.ClassNotFoundException: org.apache.beam.sdk.io.gcp.bigquery.BatchLoads$3
java.util.concurrent.ExecutionException: java.lang.RuntimeException: Error received from SDK harness for instruction -206: org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException: org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException: java.lang.IllegalArgumentException: unable to deserialize Custom Coder Bytes
...
Caused by: org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException: java.lang.IllegalArgumentException: unable to deserialize Custom Coder Bytes
...
Caused by: java.lang.IllegalArgumentException: unable to deserialize Custom Coder Bytes
...
Caused by: java.lang.ClassNotFoundException: org.apache.beam.sdk.io.gcp.bigquery.TableRowJsonCoder
...
Caused by: java.lang.RuntimeException: Error received from SDK harness for instruction -206: org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException: org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException: java.lang.IllegalArgumentException: unable to deserialize Custom Coder Bytes
...
Caused by: org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException: java.lang.IllegalArgumentException: unable to deserialize Custom Coder Bytes
...
Caused by: java.lang.IllegalArgumentException: unable to deserialize Custom Coder Bytes
...
Caused by: java.lang.ClassNotFoundException: org.apache.beam.sdk.io.gcp.bigquery.TableRowJsonCoder
...
{noformat}
----
_After you've filled out the above details, please [assign the issue to an individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist]. Assignee should [treat test failures as high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test], helping to fix the issue or find a more appropriate owner. See [Apache Beam Post-Commit Policies|https://beam.apache.org/contribute/postcommits-policies]._
> [beam_PostCommit_Java_PortabilityApi] Various GCP IO tests failing, unable to deserialize Custom DoFns and Custom Coders.
> -------------------------------------------------------------------------------------------------------------------------
>
> Key: BEAM-9745
> URL: https://issues.apache.org/jira/browse/BEAM-9745
> Project: Beam
> Issue Type: Bug
> Components: io-java-gcp, java-fn-execution, sdk-java-harness, test-failures
> Reporter: Daniel Oliveira
> Assignee: Daniel Oliveira
> Priority: Major
> Labels: currently-failing
>
> _Use this form to file an issue for test failure:_
> * [Jenkins Job|https://builds.apache.org/job/beam_PostCommit_Java_PortabilityApi/4657/]
> * [Gradle Build Scan|https://scans.gradle.com/s/c3izncsa4u24k/tests/by-project]
> Initial investigation:
> The bug appears to be popping up on BigQuery tests mostly, but also a BigTable and a Datastore test.
> Here's an example stacktrace of the two errors, showing _only_ the error messages themselves. Source: [https://scans.gradle.com/s/c3izncsa4u24k/tests/efn4wciuamvqq-ccxt3jvofvqbe]
> {noformat}
> java.util.concurrent.ExecutionException: java.lang.RuntimeException: Error received from SDK harness for instruction -191: java.lang.IllegalArgumentException: unable to deserialize Custom DoFn With Execution Info
> ...
> Caused by: java.lang.ClassNotFoundException: org.apache.beam.sdk.io.gcp.bigquery.BatchLoads$3
> ...
> Caused by: java.lang.RuntimeException: Error received from SDK harness for instruction -191: java.lang.IllegalArgumentException: unable to deserialize Custom DoFn With Execution Info
> ...
> Caused by: java.lang.ClassNotFoundException: org.apache.beam.sdk.io.gcp.bigquery.BatchLoads$3
> java.util.concurrent.ExecutionException: java.lang.RuntimeException: Error received from SDK harness for instruction -206: org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException: org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException: java.lang.IllegalArgumentException: unable to deserialize Custom Coder Bytes
> ...
> Caused by: org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException: java.lang.IllegalArgumentException: unable to deserialize Custom Coder Bytes
> ...
> Caused by: java.lang.IllegalArgumentException: unable to deserialize Custom Coder Bytes
> ...
> Caused by: java.lang.ClassNotFoundException: org.apache.beam.sdk.io.gcp.bigquery.TableRowJsonCoder
> ...
> Caused by: java.lang.RuntimeException: Error received from SDK harness for instruction -206: org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException: org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException: java.lang.IllegalArgumentException: unable to deserialize Custom Coder Bytes
> ...
> Caused by: org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException: java.lang.IllegalArgumentException: unable to deserialize Custom Coder Bytes
> ...
> Caused by: java.lang.IllegalArgumentException: unable to deserialize Custom Coder Bytes
> ...
> Caused by: java.lang.ClassNotFoundException: org.apache.beam.sdk.io.gcp.bigquery.TableRowJsonCoder
> ...
> {noformat}
> Update: Looks like this has been failing as far back as [Apr 4|https://builds.apache.org/job/beam_PostCommit_Java_PortabilityApi/4566/] after a long period where the test was consistently timing out since [Mar 31|https://builds.apache.org/job/beam_PostCommit_Java_PortabilityApi/4546/]. So it's hard to narrow down what commit may have caused this. Plus, the test was failing due to a completely different BigQuery failure before anyway, so it seems like this test will need to be completely fixed from scratch, instead of tracking down a specific breaking change.
> ----
> _After you've filled out the above details, please [assign the issue to an individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist]. Assignee should [treat test failures as high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test], helping to fix the issue or find a more appropriate owner. See [Apache Beam Post-Commit Policies|https://beam.apache.org/contribute/postcommits-policies]._
--
This message was sent by Atlassian Jira
(v8.3.4#803005)