You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Alex Amato (Jira)" <ji...@apache.org> on 2021/09/17 23:19:00 UTC
[jira] [Commented] (BEAM-12912)
:runners:direct-java:runMobileGamingJavaDirect flakey failure in
PostRelease_NightlySnapshot
[ https://issues.apache.org/jira/browse/BEAM-12912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17416964#comment-17416964 ]
Alex Amato commented on BEAM-12912:
-----------------------------------
Dataflow job id:
[https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-16_05_28_35-11695407347296868058?project=apache-beam-testing]
I believe this is the pipeline source:
https://github.com/apache/beam/blob/243128a8fc52798e1b58b0cf1a271d95ee7aa241/examples/java/src/main/java/org/apache/beam/examples/complete/game/LeaderBoard.java
writes to BQ here
[https://github.com/apache/beam/blob/243128a8fc52798e1b58b0cf1a271d95ee7aa241/examples/java/src/main/java/org/apache/beam/examples/complete/game/LeaderBoard.java#L232]
This does seem to be using CreateDisposition.CREATE_IF_NEEDED
[https://github.com/rhnagpal/Beam/blob/e99d2af023ec389e1285c28295406e94732e7a9a/BeamTutorial/src/main/java/org/apache/beam/examples/tutorial/game/utils/WriteToBigQuery.java#L119]
> :runners:direct-java:runMobileGamingJavaDirect flakey failure in PostRelease_NightlySnapshot
> --------------------------------------------------------------------------------------------
>
> Key: BEAM-12912
> URL: https://issues.apache.org/jira/browse/BEAM-12912
> Project: Beam
> Issue Type: Test
> Components: test-failures
> Reporter: Alex Amato
> Priority: P2
>
> [https://ci-beam.apache.org/job/beam_PostRelease_NightlySnapshot/1493/console]
>
> ...
> 05:41:51 org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl.insertAll(BigQueryServicesImpl.java:1054)
> 05:41:51 org.apache.beam.sdk.io.gcp.bigquery.BatchedStreamingWrite.flushRows(BatchedStreamingWrite.java:421)
> 05:41:51 org.apache.beam.sdk.io.gcp.bigquery.BatchedStreamingWrite.access$900(BatchedStreamingWrite.java:72)
> 05:41:51 org.apache.beam.sdk.io.gcp.bigquery.BatchedStreamingWrite$BatchAndInsertElements.finishBundle(BatchedStreamingWrite.java:267)
> 05:41:51 Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 404 Not Found
> 05:41:51 POST https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/datasets/beam_postrelease_mobile_gaming/tables/leaderboard_DataflowRunner_user/insertAll?prettyPrint=false
> 05:41:51 {
> 05:41:51 "code" : 404,
> 05:41:51 "errors" : [ {
> 05:41:51 "domain" : "global",
> 05:41:51 "message" : "Not found: Table apache-beam-testing:beam_postrelease_mobile_gaming.leaderboard_DataflowRunner_user",
> 05:41:51 "reason" : "notFound"
> 05:41:51 } ],
> 05:41:51 "message" : "Not found: Table apache-beam-testing:beam_postrelease_mobile_gaming.leaderboard_DataflowRunner_user",
> 05:41:51 "status" : "NOT_FOUND"
> 05:41:51 }
> 05:41:51 com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
> 05:41:51 com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:118)
> 05:41:51 com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:37)
> 05:41:51 com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:428)
> 05:41:51 com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1111)
> 05:41:51 com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:514)
> 05:41:51 com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:455)
> 05:41:51 com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:565)
> 05:41:51 org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl.lambda$insertAll$1(BigQueryServicesImpl.java:910)
> 05:41:51 org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$BoundedExecutorService$SemaphoreCallable.call(BigQueryServicesImpl.java:1594)
> 05:41:51 java.util.concurrent.FutureTask.run(FutureTask.java:266)
> 05:41:51 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> 05:41:51 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> 05:41:51 java.lang.Thread.run(Thread.java:748)
> 05:41:51 java.lang.RuntimeException: com.google.api.client.googleapis.json.GoogleJsonResponseException: 404 Not Found
> 05:41:51 POST https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/datasets/beam_postrelease_mobile_gaming/tables/leaderboard_DataflowRunner_user/insertAll?prettyPrint=false
> 05:41:51 {
> 05:41:51 "code" : 404,
> 05:41:51 "errors" : [ {
> 05:41:51 "domain" : "global",
> 05:41:51 "message" : "Not found: Table apache-beam-testing:beam_postrelease_mobile_gaming.leaderboard_DataflowRunner_user",
> 05:41:51 "reason" : "notFound"
> 05:41:51 } ],
> 05:41:51 "message" : "Not found: Table apache-beam-testing:beam_postrelease_mobile_gaming.leaderboard_DataflowRunner_user",
> 05:41:51 "status" : "NOT_FOUND"
> 05:41:51 }
> 05:41:51 org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl.insertAll(BigQueryServicesImpl.java:1001)
> 05:41:51 org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl.insertAll(BigQueryServicesImpl.java:1054)
> 05:41:51 org.apache.beam.sdk.io.gcp.bigquery.BatchedStreamingWrite.flushRows(BatchedStreamingWrite.java:421)
> 05:41:51 org.apache.beam.sdk.io.gcp.bigquery.BatchedStreamingWrite.access$900(BatchedStreamingWrite.java:72)
> 05:41:51 org.apache.beam.sdk.io.gcp.bigquery.BatchedStreamingWrite$BatchAndInsertElements.finishBundle(BatchedStreamingWrite.java:267)
> 05:41:51 Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 404 Not Found
> 05:41:51 POST https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/datasets/beam_postrelease_mobile_gaming/tables/leaderboard_DataflowRunner_user/insertAll?prettyPrint=false
> 05:41:51 {
> 05:41:51 "code" : 404,
> 05:41:51 "errors" : [ {
> 05:41:51 "domain" : "global",
> 05:41:51 "message" : "Not found: Table apache-beam-testing:beam_postrelease_mobile_gaming.leaderboard_DataflowRunner_user",
> 05:41:51 "reason" : "notFound"
> 05:41:51 } ],
> 05:41:51 "message" : "Not found: Table apache-beam-testing:beam_postrelease_mobile_gaming.leaderboard_DataflowRunner_user",
> 05:41:51 "status" : "NOT_FOUND"
> 05:41:51 }
> 05:41:51 com.google.api.client.
> 05:41:51
> 05:41:51 Waiting on bqjob_r6d256913efdd3c68_0000017beea0a437_1 ... (0s) Current status: RUNNING
> 05:41:52
> 05:41:52 Waiting on bqjob_r6d256913efdd3c68_0000017beea0a437_1 ... (0s) Current status: DONE
> 05:41:52 +------------------------------+
> 05:41:52 | user |
> 05:41:52 +------------------------------+
> 05:41:52 | user0_AsparagusBilby |
> 05:41:52 | user8_FuchsiaBilby |
> 05:41:52 | user2_BattleshipGreyCockatoo |
> 05:41:52 | user5_AmethystWombat |
> 05:41:52 | user14_ArmyGreenCaneToad |
> 05:41:52 | user2_ApricotAntechinus |
> 05:41:52 | user6_RubyNumbat |
> 05:41:52 | user7_AmethystNumbat |
> 05:41:52 | user7_AmaranthDingo |
> 05:41:52 | user5_AlmondEchidna |
> 05:41:52 +------------------------------+
> 05:41:52 Verified Almond
> 05:41:53 RUNNING_JOB=`gcloud dataflow jobs list | grep leaderboard-validation-1631795272362-386 | grep Running | cut -d' ' -f1`
> 05:41:53 if [ ! -z "${RUNNING_JOB}" ]
> 05:41:53 then
> 05:41:53 gcloud dataflow jobs cancel ${RUNNING_JOB}
> 05:41:53 else
> 05:41:53 echo "Job 'leaderboard-validation-1631795272362-386' is not running."
> 05:41:53 fi
> 05:41:53
> 05:41:53 Job 'leaderboard-validation-1631795272362-386' is not running.
> 05:41:55
> 05:41:55 **************************************
> 05:41:55 * SUCCESS: LeaderBoard successfully run on DataflowRunner. with Streaming Engine
> 05:41:55 **************************************
> 05:41:55
> 05:41:55 [SUCCESS]
> 05:41:55
> 05:41:55 FAILURE: Build failed with an exception.
> 05:41:55
> 05:41:55 * What went wrong:
> 05:41:55 Execution failed for task ':runners:direct-java:runMobileGamingJavaDirect'.
> 05:41:55 > Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
> 05:41:55
> 05:41:55 * Try:
> 05:41:55 Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
> 05:41:55
> 05:41:55 * Get more help at https://help.gradle.org
> 05:41:55
> 05:41:55 Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
> 05:41:55 Use '--warning-mode all' to show the individual deprecation warnings.
> 05:41:55 See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
> 05:41:55
> 05:41:55 BUILD FAILED in 1h 41m 35s
> 05:41:55 9 actionable tasks: 8 executed, 1 from cache
--
This message was sent by Atlassian Jira
(v8.3.4#803005)